About training FlowNet2

[zmlshiwo]

Hi, I'm a little confused about Flownet2.0 's training process. First of all, I give you my understanding of the training process.

“FlowNet2-C”: This model is first trained on FlyingChairs with "long" schedule and then fine-tuned on Things3D with fine schedule.

"FlowNet2-CS": Fixing C, then training S on FlyingChairs with "long" schedule and then fine-tuned on Things3D with fine schedule.

"FlowNet2-CSS": Fixing C and S, then training the last S on FlyingChairs with "long" schedule and then fine-tuned on Things3D with fine schedule.

"FlowNet2-CSS-ft-sd": Fine-tuning whole FlowNet2-CSS on a mixture of Things3D and CharisSHDom dataset with "fine" schedule.

"FlowNet2-SD": Training FlowNet2-SD on CharisSHDom dataset with "long" schedule.

"FlowNet2": Fixing "FlowNet2-CSS-ft-sd" and "FlowNet2-SD", training "fusion network" on FlyingChairs with "long" schedule, and then fine--tuning "fusion network" with "fine" schedule.

Is the training process correct? Especially "FlowNet2-CSS-ft-sd", "FlowNet2-SD" and "FlowNet2"

Thanks, Mingliang

[nikolausmayer]

Hi,

• FlowNet2-C/-CS/-CSS/-SD are trained as you describe

• FlowNet2-CSS-ft-sd additionally uses a nonlinear error weight (x^0.4); see supplementary material

• FlowNet2: The fusion network is trained on the same mixture as FlowNet2-CSS-ft-sd, using an accelerated schedule:

  • base learning rate 1e-4
  • gamma 0.5
  • steps at 50k, 100k, 150k, 200k, 250k
  • total training until 300k

[zmlshiwo]

@nikolausmayer Thank you very much. I have a few more questions. 1. Does the "FlowNet2-CSS-ft-sd_train.prototxt" contain the nonlinear error weight configuration?
2. In your paper, you said the "FlowNet2-CSS-ft-sd" is trained with mini-batches of 8 samples: 2 from Things3D and 6 from ChairsSDHom. How to read data from two datasets with different batch size? 3. If I want to merge two pre-trained models, “FlowNet2-CSS-ft-sd” and "FlowNet2-SD", how to do this processing?

[nikolausmayer]

Hm, I don't see the weighting function in the prototxt file. Please contact Eddy Ilg for these questions, he'll know more.

[zmlshiwo]

@eddy-ilg

[nikolausmayer]

(closed due to inactivity)

[lesurp]

Hi, sorry, yet another uninformed question here..!

How should I go with training the FlowNet2-CSS model more precisely? Does the train.prototxt handle fixing C then S (although nothing in the file hints at that, it seems to me). Or do I need to first train FlowNet2-C then Flownet2-CS etc. myself, reusing the previously computed weights (and if so, how shall I pass those weights to the next training stack)..?

[nikolausmayer]

@lesurp the provided prototxt files do not handle the stage-wise training. That you have to do yourself :)