stylizeVideo_flownet.sh cannot find FlowNet2_deploy.prototxt

[agilebean]

I got Caffe for flownet2 running on Ubuntu 16.04. Running bash stylizeVideo_flownet.sh zoo1.mp4 ./models/checkpoint-mosaic-video.t7 stops due to unfound caffemodel in the flownet2 folder:

Starting optical flow computation...
Traceback (most recent call last):
    if(not os.path.exists(args.deployproto)): raise BaseException('deploy-proto does not exist: '+args.deployproto)
BaseException: deploy-proto does not exist: /home/rstudio/flownet2/Flownet2/FlowNet2_deploy.prototxt

Looking in that directory, turns out there is not a FlowNet2_deploy.prototxt but FlowNet2_deploy.prototxt.template file. Copying the FlowNet2_deploy.prototxt.template to FlowNet2_deploy.prototxt still returns the same error message:

Starting optical flow computation...
Traceback (most recent call last):
  File "/home/rstudio/flownet2/scripts/run-flownet-many.py", line 22, in <module>
    if(not os.path.exists(args.deployproto)): raise BaseException('deploy-proto does not exist: '+args.deployproto)
BaseException: deploy-proto does not exist: /home/rstudio/flownet2/Flownet2/FlowNet2_deploy.prototxt

As before, the file does exist at precisely the path (see last line):

/flownet2/FlowNet2$ ls -al
total 638820
drwxr-xr-x  2 rstudio rstudio      4096 Apr  5 14:05 .
drwxrwxr-x 33 rstudio rstudio      4096 Apr  5 14:05 ..
-rw-rw-r--  1 rstudio rstudio     62798 Apr  5 13:58 FlowNet2_deploy.prototxt
-rw-r--r--  1 rstudio rstudio     62798 Apr 25  2017 FlowNet2_deploy.prototxt.template
-rw-rw-r--  1 rstudio rstudio     69448 Apr  5 14:05 FlowNet2_train.prototxt
-rw-r--r--  1 rstudio rstudio     69448 Apr 25  2017 FlowNet2_train.prototxt.template
-rw-r--r--  1 rstudio rstudio 653868648 Apr 25  2017 FlowNet2_weights.caffemodel.h5
rstudio@demo3:~/flownet2/FlowNet2$ pwd
/home/rstudio/flownet2/FlowNet2

So the questions are:

1. Why does the script run-flownet-multiple.sh expect a FlowNet2_deploy.prototxt and not the FlowNet2_deploy.prototxt.template? Any expection from that file that could be made explicit?
2. Why is the file (in both cases, .prototxt and .prototxt.template) not found even thought they do exist?

[agilebean]

Please help anybody - this seems to a reproducible issue after 60 trials. Case A: Using this configuration (default FlowNet2 folder) in run-flownet-multiple.sh:

chmod +x ~/flownet2/scripts/run-flownet-many.py
/home/rstudio/flownet2/scripts/run-flownet-many.py /home/rstudio/flownet2/FlowNet2/FlowNet2_weights.caffemodel.h5  /home/rstudio/flownet2FlowNet2/FlowNet2_deploy.prototxt.template  $1 --gpu ${2:-0}

makes stylizeVideo_flownet.sh return IOError: [Errno 2] No such file or directory: '/home/rstudio/flownet2FlowNet2/FlowNet2_deploy.prototxt.template'

Case B: Using this configuration (other FlowNet2 folder) in run-flownet-multiple.sh:

chmod +x ~/flownet2/scripts/run-flownet-many.py
/home/rstudio/flownet2/scripts/run-flownet-many.py /home/rstudio/flownet2/FlowNet2-KITTI/FlowNet2-KITTI_weights.caffemodel.h5  /home/rstudio/flownet2/FlowNet2-KITTI/FlowNet2-KITTI_deploy.prototxt.template  $1 --gpu ${2:-0}

makes stylizeVideo_flownet.sh return

Starting optical flow computation...
Processing tuple: ['./zoo1/frame_00003.ppm', './zoo1/frame_00002.ppm', './zoo1/flow_default/backward_3_2.flo']
Network forward pass using /home/rstudio/flownet2/FlowNet2-KITTI/FlowNet2-KITTI_weights.caffemodel.h5.
Succeeded.
Processing tuple: ['./zoo1/frame_00003.ppm', './zoo1/frame_00004.ppm', './zoo1/flow_default/forward_3_4.flo']
run-flownet-multiple.sh: line 9: 23691 Killed                  /home/rstudio/flownet2/scripts/run-flownet-many.py /home/rstudio/flownet2/FlowNet2-KITTI/FlowNet2-KITTI_weights.caffemodel.h5 /home/rstudio/flownet2/FlowNet2-KITTI/FlowNet2-KITTI_deploy.prototxt.template $1 --gpu ${2:-0}

So Case B means two things:

1. it is something about the prototxt.template file in the FlowNet2 folder that is different.
2. the root cause why the process was killed could be the interaction with --gpu ${2:-0}?

In any case, could you post the sample code of run-flownet-multiple.sh for several cases of calling correctly the prototxt.template?

[agilebean]

Another hint I wish somebody could investigate soon: run-flownet-multiple.sh calls the python script flownet2/run-flownet-multiple.py that parses the arguments like this:

parser = argparse.ArgumentParser()
parser.add_argument('caffemodel', help='path to model')
parser.add_argument('deployproto', help='path to deploy prototxt template')
parser.add_argument('listfile', help='one line should contain paths "img0.ext img1.ext out.flo"')
parser.add_argument('--gpu',  help='gpu id to use (0, 1, ...)', default=0, type=int)
parser.add_argument('--verbose',  help='whether to output all caffe logging', action='store_true')

So actually, it expects a 'listfile' which is not in the fast-artistic-videos/run-flownet-multiple.sh, there only the arguments for caffemodel and deployproto:

.../scripts/run-flownet-many.py .../FlowNet2_weights.caffemodel.h5 .../FlowNet2_deploy.prototxt $1 --gpu ${2:-0}

Is this the root cause of the process being killed? Again, please provide sample code for fast-artistic-videos/run-flownet-multiple.sh - THANKS!

[manuelruder]

I can only assent what nikolausmayer said here.

The main reason why researchers like me provide source code at all is mainly for other researchers who want to reproduce the results or continue research in that area. If you want to use that code, it's your responsibility to gets things to run. My code is fully accessible, so you can change whatever you need to change in the code to get it to run on your system, but I can't help you with that.

If you need support stylizing a video, there is deepart.io, where you can submit a video and then we process it on our servers for you. It's not for free though.

[agilebean]

I fully understand. And in general, I do agree! It's not a black-and-white though. Sometimes, a little hint like nikolausmayer gave in your referenced link does a help a lot - a little effort for him but a big benefit whoever runs into the same problem. So in the above issue, is it too much to give one example of run-flownet-multiple.sh ? Maybe it is. But that would be enough to clarify whether it is a path problem I have...

[manuelruder]

The sample code is provided by my scripts. Exactly for that reason I added the scripts so that others can see how the pipeline is called. run-flownet-multiple.sh is called by makeOptFlow_flownet.sh, in the following way:

flowCommandLine="bash run-flownet-multiple.sh"
eval $flowCommandLine ${folderName}/flow_list.txt

(where you can combine the two into a single line of course)

[agilebean]

Thanks. I really don't want to bother with more questions and I did have a look at your scripts- the problem causing all my questions is that it only specifies "corresponding directories" which is unclear you must admit. Is it ok if I replace ... with ~/flownet2 if there is my flownet2 installation? That is the only question you need to answer.

Thanks for the calling function. I did find out that The call of the script seems to be no problem as the caffe model is found. It does not find the other arguments.

[manuelruder]

"It does not find the other arguments."

Normally, $1 should be replaced with the first command line argument. But this didn't happen in your case? This may be another peculiarity of your environment and a starting point for your research. I can assure you that the code has been tested on my machine and if there were such basic mistakes like wrong or missing arguments I would have noticed.