search

drinkingcoder / FlowFormer-Official

Apache License 2.0
406 stars 47 forks source link

Training on a custom dataset #14

Closed owen24819 closed 2 years ago

owen24819 commented 2 years ago

Hi @drinkingcoder,

Really nice work! Really cool to see the use of transformers in optical flow. I have more of a question than an issue. I am interested in training FlowFormer on my own dataset. Is it possible create your own dataset? If so, can you give any recommendations / tools to help curate your own dataset? For context, I know very little about optical flow; I just learned about the field last month.

Thanks, Owen

drinkingcoder commented 2 years ago

To the best of my knowledge, there may be two solutions to create datasets for optical flow: 1) Rendering synthetic images and obtaining GT flows via the renderer. (FlyingChairs, Things, and Sintel) 2) Transferring depth to flow with camera poses. (KITTI and MegaDepth). However, this strategy requires the scene is almost rigid and dense depth is not cheap. Directly annotating pixel-wise flow via humans is also too expensive. Our arxiv pre-print LIFE is devoted to partially relieve learning flow from such expensive training data, i.e., weakly supervised flow learning. Hope this paper can help you.

owen24819 commented 2 years ago

I appreicate the response. I will check out your other paper!