search

lmb-freiburg / flownet2

FlowNet 2.0: Evolution of Optical Flow Estimation with Deep Networks
https://lmb.informatik.uni-freiburg.de/Publications/2017/IMKDB17/
Other
1k stars 318 forks source link

Simple way to generate ground-truth from a given video. #108

Closed ailgun closed 6 years ago

ailgun commented 6 years ago

Hello,

First of all sorry for the nativity of the following message, I hope you won’t mind helping me out on this.

We want to use this aesthetic of FlowNet on a small art project which is not commercial at all and even a personal endeavour. I’m quite quite good with Python and coding but could not compile Caffe and getting this error even though we are pretty sure scipy is installed and running when we run the code;

Traceback (most recent call last): File "/Users/xxx/xxx/opticalflow/scripts/run-flownet.py", line 7, in from scipy import misc ImportError: No module named scipy

But our question is that even if we could run the code on multiple images does this code itself produce the video file for ground truth? To be more clear; would it be enough to feed each frame of a video for it to create the ground truth? If not, as it’s all we need, is there a simpler way of creating this colourful ground-truth of any given video, maybe like shown here; https://www.youtube.com/watch?v=kT3uAHTpn8A

Also, we feel like we failed to compile it to even start the process, I wonder where can we get a more in depth information how to compile.

I can maybe try the web-cam version as well if I can find the source code for it.

Again, as I said earlier, sorry for the noob question or wish but if you would want to help out someone who just wishes to try this in a simple way for the ground-truth images I would be beyond grateful!

Best wishes and thanks for the inspiring work!

nikolausmayer commented 6 years ago

Hi,

if you don't mind installing Docker on your machine (and the additional nvidia-docker), I'd recommend using our Docker image for FlowNet2. This will make compiling FlowNet2 a no-brainer. Otherwise, our Dockerfile tells you which packages you need to run FlowNet2. In your case, at least python-scipy might be missing.

If you have a sequence of images A, B, C, D.... then our run-flownet-many.py script will produce an optical flow file for each pair. However, these are not yet the colorful images you want -- you have to do the conversion yourself. The reference code for this can be found [here](http://vision.middlebury.edu/flow/data/ ("flow-code.zip" for C++, "flow-code-matlab.zip" for Matlab code). The resulting images can be put together into a video file, e.g. using ffmpeg.

I'm sorry, but code for the webcam application is not available online (but it doesn't do much more than what I wrote above).

ailgun commented 6 years ago

Hello,

Thanks so much for the answer. No no I don't mind at all but using a mac with a non-NVIDIA card I'll try it on a friends laptop. Well, I have tried importing scipy and misc from it as well on a seperate window and it works - but somehow within the context of this .py file it fails to import misc from scipy. Will try to resolve that as well.

I see, it seems I'll be able to can manage to create the .flo files but I literally have no experience with C++ or matlab, maybe I'll just ask a computer engineer friend when I successfully reach at that point!

Thanks again, I think I'll let know how it goes soon! Best wishes!