search

hassony2 / kinetics_i3d_pytorch

Inflated i3d network with inception backbone, weights transfered from tensorflow
MIT License
528 stars 115 forks source link

Details regarding optical flow pre-processing #19

Open simonguiroy opened 6 years ago

simonguiroy commented 6 years ago

Hi Yana! Do you have some details on the your pre-processing for the optical flow? I've tried using your model with my own pre-processing that worked with the original TensorFlow repo of I3D, but when using your implementation of I3D the predictions are totally off when I use the flow stream.

hassony2 commented 6 years ago

Hi Simon,

As you might see here, I don't do anything specific beyond loading their sample in my code.

I therefore don't know what might cause this discrepancy.

Let me know if you find out where that came from !

Yana

simonguiroy commented 5 years ago

Hi Yana,

I encountered another problem when using your code. I after creating the model, if model.train() is called, training accuracy drops significantly, compared to when model.eval() is called, even without with torch.no_grads(). Could there be something with the batchnorm layer implementation, that is linked to that behavior?

Thanks again

Simon

On Thu, 6 Sep 2018 at 12:12, Yana notifications@github.com wrote:

Hi Simon,

As you might see here https://github.com/hassony2/kinetics_i3d_pytorch/blob/a4b815fb95d311bbf01734f4d1d3610f7f39cde1/i3d_pt_demo.py#L35, I don't do anything specific beyond loading their sample in my code.

I therefore don't know what might cause this discrepancy.

Let me know if you find out where that came from !

Yana

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/hassony2/kinetics_i3d_pytorch/issues/19#issuecomment-419153002, or mute the thread https://github.com/notifications/unsubscribe-auth/APn0cDhMjGMbXVAgM5KuKSlxB4DwlrMJks5uYUmHgaJpZM4WdODu .

gurkirt commented 5 years ago

It is probably due to batch normalisation layers acting up. If you want to evaluate new videos then use .eval(), If you want to train the model with a smaller batch size then freeze the normalisation layer.

ed-fish commented 4 years ago

See this issue which details optical flow pre-processing on the original Tensorflow implementation. https://github.com/deepmind/kinetics-i3d/issues/87