capture image after augmentation give black image

[alhersh]

Hello,

I am trying to capture the images after performing augmentation, but the images that I capture are all black! I have used the ImgWriter to capture them.

Any suggestion?

thanks.

[nikolausmayer]

The input data is normalized to a [0,1] range (using Eltwise layers) before the augmentation layers. If you do not undo this scaling, your images are likely just far too dark :slightly_smiling_face:

[alhersh]

Thanks for the reply and the clarification Nikolaus. But, what is the purpose of normalizing images to [0,1] range? And how this affects the augmentation and the predicted optical flow? Can we do augmentation without normalizing the input?

[alhersh]

Just now, I have removed the Eltwise layers before the augmentation and still I am getting black images!

[nikolausmayer]

I don't think removing the pre-augmentation Eltwise layers will work. The augmentation layers expect the [0,1] data range and probably don't know how to deal with other data. The normalization is likely not really necessary for the network (meaning that you could retrain without it and it would still work), but our nets have been trained with it... so now they need the normalization at test time, too :slightly_smiling_face: I suggest adding another Eltwise layer between the augmentation and your ImgWriter, to "denormalized" the augmented data. The new layer should just multiply everything with 255.

[alhersh]

I have did as you have suggested and that works perfectly, thanks _, I have tried something else, I have changed the scale in the ImgWriter writer_param to scale: 255, it seems to produce slimier result, but what do you think? Does this do the same as adding Eltwise to multiply by 255?

Another thing, I am building LMDB for the images data, I have converted the images to PPM using linux convert command. Is it necessary to Eltwise (normalize) the images again using the Eltwise layer before the augmentation layer which has the following: eltwise_param { operation: SUM coeff: 0.00392156862745098 } Moreover, as you have mentioned that you are using 768 x 320 cropping for Sintel, but the actual image size is 1024 x 436, you can use 1024 x 384 or 960 x 384. Why you used the mentioned cropping (768 x 320)?

Thanks.

[nikolausmayer]

I think doing the scaling in the ImgWriter layer should work, too. Perhaps there's a different in how edge cases are treated, but it sounds like it should do the same.

If the augmentation layer includes this scaling, then I think you don't have to do any other scaling.

We tried to strike a balance between "larger crops → better" and "smaller crops → more possible different training samples".

[alhersh]

Many thanks for your patience Nikolaus in replying all my questions and for all the clarifications.