Closed Jar7 closed 5 years ago
1, We failed to train one model to work fine for many other datasets.
2, SBR could help in this case, but we found that SBR is sensitive to many aspects, such as hyper-parameters and resolution. It would cost much time to tune it in another dataset/model.
3, Since different videos may have different LK iterations to converge, it is not that straightforward to implement it by batch.
Thank you for your reply!
You need to retrain the model on another video. If you want to use the trained model on any video, it requires to train this model on a very large number of videos.
Originally posted by @D-X-Y in https://github.com/facebookresearch/supervision-by-registration/issues/40#issuecomment-475639856
Hi @Jar7 , 1, Yes, training the model with SBR on the target video is necessary to get a visually good effect.
2, It is hard to give a precise hyper-parameter for me right now. In my experience, I would debug to see the intermedia LK results on the training data, and see whether these hyper-parameters cause many failure cases or not.
Hi @Jar7 , 1, Yes, training the model with SBR on the target video is necessary to get a visually good effect.
2, It is hard to give a precise hyper-parameter for me right now. In my experience, I would debug to see the intermedia LK results on the training data, and see whether these hyper-parameters cause many failure cases or not.
Thank you!
Hi @D-X-Y, Excuse me, May I have further questions..
I found that the pretrained model cpm_vgg16-epoch-049-050.pth output stable landmarks for demo-sbr.mp4. Has the demo-sbr,mp4 been trained? How many epoch should be trained for each video?
Have you found why a model for any video cannot be trained? Is it because the lk operation is sensitive to the video? If the lkloss is changed to another opt-flow loss, whether another additive loss would help to stabilize the output of backbone?
Hello, I understand the model is designed to detect landmarks for a video which has been trained. However, I am wondering whether I can train a model with large scale dataset to do inference for any dataset, so that the model can do runtime inference.
May I have some questions:
Thank you.