search

lmb-freiburg / flownet2

FlowNet 2.0: Evolution of Optical Flow Estimation with Deep Networks
https://lmb.informatik.uni-freiburg.de/Publications/2017/IMKDB17/
Other
1k stars 318 forks source link

About the GenerateAugmentationParameters layer #20

Closed thbupt closed 7 years ago

thbupt commented 7 years ago

I have some questions about the GenerateAugmentationParameters layer. In the following FlowNet2.prototxt, it seems that this layer can generate more augmentation parameters than DataAugmentation layer if the aug.mode() == "add", such as the brightness, gamma, constract and color, however, I think they can also be generated in the DataAugmentation layer, so why not use the DataAugmentation layer to generate all the parameters? Besides in the FlowNetC.prototxt, I found the spatial transforming paramters (rotate, translate, zoom )are both generated by the two layers. If the aug.mode() == "add", it seems there are two different groups of spatial transforming paramters in the output blob (that is blob8 in the FlowNetC.prototxt).

FlowNet2.prototxt layer { name: "img0s_aug" type: "DataAugmentation" bottom: "blob13" top: "img0_aug" top: "blob16" augmentation_param { max_multiplier: 1 augment_during_test: false recompute_mean: 1000 mean_per_pixel: false translate { rand_type: "uniform_bernoulli" exp: false mean: 0 spread: 0.2 prob: 1.0 } lmult_pow { rand_type: "uniform_bernoulli" exp: true mean: -0.2 spread: 0.4 prob: 1.0 } lmult_mult { rand_type: "uniform_bernoulli" exp: true mean: 0.0 spread: 0.4 prob: 1.0 } lmult_add { rand_type: "uniform_bernoulli" exp: false mean: 0 spread: 0.03 prob: 1.0 } sat_pow { rand_type: "uniform_bernoulli" exp: true mean: 0 spread: 0.4 prob: 1.0 } sat_mult { rand_type: "uniform_bernoulli" exp: true mean: -0.3 spread: 0.5 prob: 1.0 } sat_add { rand_type: "uniform_bernoulli" exp: false mean: 0 spread: 0.03 prob: 1.0 } col_pow { rand_type: "gaussian_bernoulli" exp: true mean: 0 spread: 0.4 prob: 1.0 } col_mult { rand_type: "gaussian_bernoulli" exp: true mean: 0 spread: 0.2 prob: 1.0 } col_add { rand_type: "gaussian_bernoulli" exp: false mean: 0 spread: 0.02 prob: 1.0 } ladd_pow { rand_type: "gaussian_bernoulli" exp: true mean: 0 spread: 0.4 prob: 1.0 } ladd_mult { rand_type: "gaussian_bernoulli" exp: true mean: 0.0 spread: 0.4 prob: 1.0 } ladd_add { rand_type: "gaussian_bernoulli" exp: false mean: 0 spread: 0.04 prob: 1.0 } col_rotate { rand_type: "uniform_bernoulli" exp: false mean: 0 spread: 1 prob: 1.0 } crop_width: 448 crop_height: 320 chromatic_eigvec: 0.51 chromatic_eigvec: 0.56 chromatic_eigvec: 0.65 chromatic_eigvec: 0.79 chromatic_eigvec: 0.01 chromatic_eigvec: -0.62 chromatic_eigvec: 0.35 chromatic_eigvec: -0.83 chromatic_eigvec: 0.44 noise { rand_type: "uniform_bernoulli" exp: false mean: 0.03 spread: 0.03 prob: 1.0 } } } layer { name: "aug_params1" type: "GenerateAugmentationParameters" bottom: "blob16" bottom: "blob13" bottom: "img0_aug" top: "blob17" augmentation_param { augment_during_test: false gamma { rand_type: "gaussian_bernoulli" exp: true mean: 0 spread: 0.02 prob: 1.0 } brightness { rand_type: "gaussian_bernoulli" exp: false mean: 0 spread: 0.02 prob: 1.0 } contrast { rand_type: "gaussian_bernoulli" exp: true mean: 0 spread: 0.02 prob: 1.0 } color { rand_type: "gaussian_bernoulli" exp: true mean: 0 spread: 0.02 prob: 1.0 } } coeff_schedule_param { half_life: 50000 initial_coeff: 0.5 final_coeff: 1 } }

FlowNetC.prototxt layer { name: "img0s_aug" type: "DataAugmentation" bottom: "blob4" top: "img0_aug" top: "blob7" propagate_down: false augmentation_param { max_multiplier: 1 augment_during_test: false recompute_mean: 1000 mean_per_pixel: false translate { rand_type: "uniform_bernoulli" exp: false mean: 0 spread: 0.4 prob: 1.0 } rotate { rand_type: "uniform_bernoulli" exp: false mean: 0 spread: 0.4 prob: 1.0 } zoom { rand_type: "uniform_bernoulli" exp: true mean: 0.2 spread: 0.4 prob: 1.0 } squeeze { rand_type: "uniform_bernoulli" exp: true mean: 0 spread: 0.3 prob: 1.0 } lmult_pow { rand_type: "uniform_bernoulli" exp: true mean: -0.2 spread: 0.4 prob: 1.0 } lmult_mult { rand_type: "uniform_bernoulli" exp: true mean: 0.0 spread: 0.4 prob: 1.0 } lmult_add { rand_type: "uniform_bernoulli" exp: false mean: 0 spread: 0.03 prob: 1.0 } sat_pow { rand_type: "uniform_bernoulli" exp: true mean: 0 spread: 0.4 prob: 1.0 } sat_mult { rand_type: "uniform_bernoulli" exp: true mean: -0.3 spread: 0.5 prob: 1.0 } sat_add { rand_type: "uniform_bernoulli" exp: false mean: 0 spread: 0.03 prob: 1.0 } col_pow { rand_type: "gaussian_bernoulli" exp: true mean: 0 spread: 0.4 prob: 1.0 } col_mult { rand_type: "gaussian_bernoulli" exp: true mean: 0 spread: 0.2 prob: 1.0 } col_add { rand_type: "gaussian_bernoulli" exp: false mean: 0 spread: 0.02 prob: 1.0 } ladd_pow { rand_type: "gaussian_bernoulli" exp: true mean: 0 spread: 0.4 prob: 1.0 } ladd_mult { rand_type: "gaussian_bernoulli" exp: true mean: 0.0 spread: 0.4 prob: 1.0 } ladd_add { rand_type: "gaussian_bernoulli" exp: false mean: 0 spread: 0.04 prob: 1.0 } col_rotate { rand_type: "uniform_bernoulli" exp: false mean: 0 spread: 1 prob: 1.0 } crop_width: 448 crop_height: 320 chromatic_eigvec: 0.51 chromatic_eigvec: 0.56 chromatic_eigvec: 0.65 chromatic_eigvec: 0.79 chromatic_eigvec: 0.01 chromatic_eigvec: -0.62 chromatic_eigvec: 0.35 chromatic_eigvec: -0.83 chromatic_eigvec: 0.44 noise { rand_type: "uniform_bernoulli" exp: false mean: 0.03 spread: 0.03 prob: 1.0 } } } layer { name: "aug_params1" type: "GenerateAugmentationParameters" bottom: "blob7" bottom: "blob4" bottom: "img0_aug" top: "blob8" augmentation_param { augment_during_test: false translate { rand_type: "gaussian_bernoulli" exp: false mean: 0 spread: 0.03 prob: 1.0 } rotate { rand_type: "gaussian_bernoulli" exp: false mean: 0 spread: 0.03 prob: 1.0 } zoom { rand_type: "gaussian_bernoulli" exp: true mean: 0 spread: 0.03 prob: 1.0 } gamma { rand_type: "gaussian_bernoulli" exp: true mean: 0 spread: 0.02 prob: 1.0 } brightness { rand_type: "gaussian_bernoulli" exp: false mean: 0 spread: 0.02 prob: 1.0 } contrast { rand_type: "gaussian_bernoulli" exp: true mean: 0 spread: 0.02 prob: 1.0 } color { rand_type: "gaussian_bernoulli" exp: true mean: 0 spread: 0.02 prob: 1.0 } } coeff_schedule_param { half_life: 50000 initial_coeff: 0.5 final_coeff: 1 } }

eddy-ilg commented 7 years ago

Hi, the DataAugmentation layer generates parameters from scratch and directly augments data. The GenerateAugmentationParameters layer only generates parameters. It can additionally take existing parameters and modify them by adding transformations.