search

visinf / irr

Iterative Residual Refinement for Joint Optical Flow and Occlusion Estimation (CVPR 2019)
Apache License 2.0
192 stars 33 forks source link

how to choose VALIDATION_INDICES ? #47

Closed poincarelee closed 2 years ago

poincarelee commented 2 years ago

Hi, in sintel.py you fixed the VALIDATION_INDICES as follows:

image I was wondering if these values are chosen meaningfully. Means five clips? What will happen if I want to choose indices randomly?

hurjunhwa commented 2 years ago

Hi, I followed the train/val split from FlowNet2 (https://github.com/lmb-freiburg/flownet2/issues/141) for a fair comparison. You can freely change if you want. If there are appearance overlaps between train and val split, it may not be easy to find a good early stopping point.

poincarelee commented 2 years ago

Thanks a lot. Seems that if much more data is used in the model, validation indices are hard to choose as fixed values due to the proportion. I was wondering if the choice of these indices were needed to keep as successive or more precisely part of a whole action. Would it influence the performance of model when indices are chosen randomly?

hurjunhwa commented 2 years ago

In that case, How about using the k-fold cross-validation instead of using the fixed validation set? In the end, what matters is to find a good stopping point that generalizes well to the test set, not to the validation set.

poincarelee commented 2 years ago

Ok, you are right.