Open santurini opened 1 year ago
Hi, To train your own optical flow model using the approach in this paper, you can follow these steps:
Please note that we don't provide the training code for optical flow models. You can refer to the training code of popular models like RAFT or GMA to train your own optical flow models and just modify a dataloader code for reading your generated data into the training scripts.
Thank you very much! This means I can generate a huge dataset simply by stacking together multiple videos and pass the in the form of (frame1, frame2) right?
But shouldn't be the performance be limited by the goodness of the model? I mean if I train my model on data generated by the model itself how should this help in fine-tuning the same model?
Hello, I would like to train my own optical flow using this scheme, what changes should I do ninth RealFlow.py script? Reading the paper I understood that only the Optical Flow gets trained while DPT and BHF are fixed. This means I should remove torch.no_grad() for the flow_estimation part but which loss should I use to backpropagate? Computed between which pair of tensors?
Really look forward to know better this work, I really like it!