search

yjxiong / temporal-segment-networks

Code & Models for Temporal Segment Networks (TSN) in ECCV 2016
BSD 2-Clause "Simplified" License
1.53k stars 477 forks source link

Help with Optical Flow #23

Closed praneetha111 closed 7 years ago

praneetha111 commented 7 years ago

Hi,

I have looked at the optical flow extraction in the denseflow code and I see that TVL1 algorithm is used. But Farneback is much better in performance compared to TVL1. Did you happen to run any tests to compare the accuracy in both the cases?

Thanks

wanglimin commented 7 years ago

Hi,

Where did you get the conclusion of "But Farneback is much better in performance compared to TVL1."?

But we did not try Farneback optical flow.

Limin

praneetha111 commented 7 years ago

I have tried to run experiments on my end. I might be wrong.

But using Nvidia TitanX 12GB gpu, Farneback took around 14ms and TV-L1 took around 22ms for the same video.

I am trying to do a real time video recognition, so even millisecond would matter.

On

yjxiong commented 7 years ago

Hi @praneetha111 ,

We haven't tested the performance of Farneback flow yet. So we encourage you to give it a try and share the results with us and other users.

Just a grain of confidence, we have tested TSN on RGB difference frames. This simple motion representation can already give reasonable accuracy. So I think Farnback at least should not be much worse than TV-L1 in terms of recognition accuracy.

praneetha111 commented 7 years ago

Hi,

Thank you very much for the reply. I will try with Farneback and post the results. Is there any specific reason you chose TV-L1 among all the existing optical flow algorithms?

Also, do you think introducing LSTM component to this framework would help in boosting the accuracy a bit more. Would that be worth trying? and is the RGB difference modality included in the published repository?

Thanks again for your patient replies. Really appreciate it

yjxiong commented 7 years ago

No special reason, we started with TVL1 long time ago mainly due to its reasonable run time. RGB difference, together with the warp flow modality, will be added to this repo in near future.

Thank you in advance for posting your results so others may benefit from it.

praneetha111 commented 7 years ago

Ok Thank you for the reply.