search

feihuzhang / SeparableFlow

Separable Flow: Learning Motion Cost Volumes for Optical Flow Estimation
MIT License
62 stars 9 forks source link

GPU memory usage #4

Open az-ja opened 2 years ago

az-ja commented 2 years ago

Dear Feihu,

Thank you very much for sharing your precious work with us!

Since in the paper SeparableFlow it is mentioned that this approach reduces the memory usage significantly due to incorporating two samller 3D cost volume instead of a large 4D cost volume, I wanted to compare the memory usage to RAFT. I trained both methods on FlyingChairs with batch size of 10, using mixed precision with the same patch size (320, 448). RAFT needed 10GBs of RAM on the GPU and Seperable flow needed 20GBs. Could you give me some hint why Sparable flow is using so much RAM on the GPU?

I appreciate any hint.

Sincerely, Azin

feihuzhang commented 2 years ago

Currently, the code doesn't support mixed-precision training.

The memory consumption is higher than RAFT as we use the memory to store the 4D correlation volume for the attention step that is a little different from the original paper (which do not store the correlation volume). However, this new implementation can improve the accuracy.

az-ja commented 2 years ago

Thank you for your answer. Could you please provide a commit or a version that implements the paper exactly? Or could you be more detailed what exactly I can do to down grade the implementation to the paper's exact implementation? The more effective use of memory of the original paper and still being able to attain such great results is the reason I am so interested in SeparableFlow. Thank you very much for your help.