Flownet Docker steps

[kboruff]

Hello,

I'm aware you explicitly state not to ask about setting up FlowNet, but I saw a step in your setup that I'm hoping you can clarify.

"There is also a Dockerfile for easy installation of the complete code in one step: flownet2-docker (GitHub)"

Golden, got it cooching with Nvidia-Docker 2, I can run this script long hand for two files at a time. I call it with "./run-network.sh -n FlowNet2 -v data/0000000-imgL.png data/0000001-imgL.png flow.flo"

"Then edit run-flownet-multiple.sh and set the paths to the FlowNet executable, model definition and pretrained weights."

Opening that file, it appears that it is looking for a link to the stuff that is inside the Docker container. I've tried a local installation about 15 times, so I'm thinking the Docker container isn't just the easiest, but maybe the ONLY way to get it working.

So, how did you find the paths inside the Docker container for the executable, models defs, and weights?

[manuelruder]

I can't help you with this. I never tried the Docker image or installed FlowNet myself, instead I used the binaries that were already on our university computer.

I will leave this open though, maybe someone else has solved this already.

[positlabs]

I'm in the process of building a docker container to run fast-artistic-videos. The approach I'm taking is to use the flownet2 image as the base in my container definition. That way I can access the flownet dir at /flownet2

First, build the flownet2 image, then declare it as the base in your container def: FROM flownet2

Then you can copy your customized run-flownet-multiple.sh script to overwrite the clone. It should look like this:

chmod +x /flownet2/flownet2/scripts/run-flownet-many.py
/flownet2/flownet2/scripts/run-flownet-many.py /flownet2/flownet2/models/FlowNet2/FlowNet2_weights.caffemodel.h5 /flownet2/flownet2/models/FlowNet2/FlowNet2_deploy.prototxt $1 --gpu ${2:-0}

[oleprinds]

If you want to use the flownet docker image, but not build your own image on top, you can modify the script to start the docker image and execute the command, similar to what happens in run-network.sh example that comes with the flownet docker image.

I used the following script

#!/bin/bash 

# You may want to set this if it does not find your caffe installation
#export CAFFE_BIN=.../caffe.bin

BASEDIR="/flownet2/flownet2/models";
WORKDIR="/flownet2/flownet2/scripts";
NETWORK="FlowNet2";

WEIGHTS="${BASEDIR}/${NETWORK}/${NETWORK}_weights.caffemodel*";
DEPLOYPROTO="${BASEDIR}/${NETWORK}/${NETWORK}_deploy.prototxt.template";

#echo $PWD

nvidia-docker run \
  --rm \
  --volume "${PWD}:/input-output:rw" \
  --workdir "${WORKDIR}" \
  -it flownet2 /bin/bash -c "cd ..; source set-env.sh; cd -; python /input-output/run-flownet-many-docker.py ${WEIGHTS} ${DEPLOYPROTO} $1 --gpu ${2:-0}"

and then I put a modified version of run-flownet-many.py in the root of fast-artistic-videos with the name 'run-flownet-many-docker.py'

#!/usr/bin/env python2.7

from __future__ import print_function

import os, sys, numpy as np
import argparse
from scipy import misc
import caffe
import tempfile
from math import ceil

def dockerize_filepath(path):
    """
    Convert a given filepath to be relative to the input-output folder that this
    container gets from the host system.
    """
    return os.path.join('/input-output', path)

parser = argparse.ArgumentParser()
parser.add_argument('caffemodel', help='path to model')
parser.add_argument('deployproto', help='path to deploy prototxt template')
parser.add_argument('listfile', help='one line should contain paths "img0.ext img1.ext out.flo"')
parser.add_argument('--gpu',  help='gpu id to use (0, 1, ...)', default=0, type=int)
parser.add_argument('--verbose',  help='whether to output all caffe logging', action='store_true')

args = parser.parse_args()

if(not os.path.exists(dockerize_filepath(args.caffemodel))): raise BaseException('caffemodel does not exist: '+args.caffemodel)
if(not os.path.exists(dockerize_filepath(args.deployproto))): raise BaseException('deploy-proto does not exist: '+args.deployproto)
if(not os.path.exists(dockerize_filepath(args.listfile))): raise BaseException('listfile does not exist: '+args.listfile)

def readTupleList(filename):
    list = []
    for line in open(dockerize_filepath(filename)).readlines():
        if line.strip() != '':
            list.append(line.split())

    return list

ops = readTupleList(args.listfile)

width = -1
height = -1

for ent in ops:
    print('Processing tuple:', ent)

    num_blobs = 2
    input_data = []
    img0 = misc.imread(dockerize_filepath(ent[0]))
    if len(img0.shape) < 3: input_data.append(img0[np.newaxis, np.newaxis, :, :])
    else:                   input_data.append(img0[np.newaxis, :, :, :].transpose(0, 3, 1, 2)[:, [2, 1, 0], :, :])
    img1 = misc.imread(dockerize_filepath(ent[1]))
    if len(img1.shape) < 3: input_data.append(img1[np.newaxis, np.newaxis, :, :])
    else:                   input_data.append(img1[np.newaxis, :, :, :].transpose(0, 3, 1, 2)[:, [2, 1, 0], :, :])

    if width != input_data[0].shape[3] or height != input_data[0].shape[2]:
        width = input_data[0].shape[3]
        height = input_data[0].shape[2]

        vars = {}
        vars['TARGET_WIDTH'] = width
        vars['TARGET_HEIGHT'] = height

        divisor = 64.
        vars['ADAPTED_WIDTH'] = int(ceil(width/divisor) * divisor)
        vars['ADAPTED_HEIGHT'] = int(ceil(height/divisor) * divisor)

        vars['SCALE_WIDTH'] = width / float(vars['ADAPTED_WIDTH']);
        vars['SCALE_HEIGHT'] = height / float(vars['ADAPTED_HEIGHT']);

        tmp = tempfile.NamedTemporaryFile(mode='w', delete=False)

        proto = open(args.deployproto).readlines()
        for line in proto:
            for key, value in vars.items():
                tag = "$%s$" % key
                line = line.replace(tag, str(value))

            tmp.write(line)

        tmp.flush()

    if not args.verbose:
        caffe.set_logging_disabled()
    caffe.set_device(args.gpu)
    caffe.set_mode_gpu()
    net = caffe.Net(tmp.name, args.caffemodel, caffe.TEST)

    input_dict = {}
    for blob_idx in range(num_blobs):
        input_dict[net.inputs[blob_idx]] = input_data[blob_idx]

    #
    # There is some non-deterministic nan-bug in caffe
    #
    print('Network forward pass using %s.' % args.caffemodel)
    i = 1
    while i<=5:
        i+=1

        net.forward(**input_dict)

        containsNaN = False
        for name in net.blobs:
            blob = net.blobs[name]
            has_nan = np.isnan(blob.data[...]).any()

            if has_nan:
                print('blob %s contains nan' % name)
                containsNaN = True

        if not containsNaN:
            print('Succeeded.')
            break
        else:
            print('**************** FOUND NANs, RETRYING ****************')

    blob = np.squeeze(net.blobs['predict_flow_final'].data).transpose(1, 2, 0)

    def readFlow(name):
        if name.endswith('.pfm') or name.endswith('.PFM'):
            return readPFM(name)[0][:,:,0:2]

        f = open(name, 'rb')

        header = f.read(4)
        if header.decode("utf-8") != 'PIEH':
            raise Exception('Flow file header does not contain PIEH')

        width = np.fromfile(f, np.int32, 1).squeeze()
        height = np.fromfile(f, np.int32, 1).squeeze()

        flow = np.fromfile(f, np.float32, width * height * 2).reshape((height, width, 2))

        return flow.astype(np.float32)

    def writeFlow(name, flow):
        f = open(name, 'wb')
        f.write('PIEH'.encode('utf-8'))
        np.array([flow.shape[1], flow.shape[0]], dtype=np.int32).tofile(f)
        flow = flow.astype(np.float32)
        flow.tofile(f)

    writeFlow(dockerize_filepath(ent[2]), blob)

The key is simply to 'dockerize' the paths in the script, so that it uses the volume 'input-output'

[bafonso]

I was then able to build the flownet2-docker with the older cuda and cudnn but I'm getting a problem with google protobuf. This is simply if I ssh into the docker.. I get the same if I run the demo run flownet2-docker script.

>>> import caffe
libdc1394 error: Failed to initialize libdc1394
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/flownet2/flownet2/python/caffe/__init__.py", line 1, in <module>
    from .pycaffe import Net, SGDSolver, NesterovSolver, AdaGradSolver, RMSPropSolver, AdaDeltaSolver, AdamSolver
  File "/flownet2/flownet2/python/caffe/pycaffe.py", line 15, in <module>
    import caffe.io
  File "/flownet2/flownet2/python/caffe/io.py", line 8, in <module>
    from caffe.proto import caffe_pb2
  File "/flownet2/flownet2/python/caffe/proto/caffe_pb2.py", line 10, in <module>
    from google.protobuf import symbol_database as _symbol_database
ImportError: cannot import name symbol_database
>>> exit

[positlabs]

I fixed this by upgrading protobuf

RUN curl "https://bootstrap.pypa.io/get-pip.py" -o "get-pip.py" && \
    python get-pip.py && \
    pip install --upgrade protobuf --ignore-installed six && \
    rm get-pip.py

[bafonso]

Thanks @oleprinds , that script works, I am now running some optical flow on a video I've been using. Seems like it's using the GPU as well as 100% of a core on my CPU but definitely not as much GPU as I'd expect, seems like it's a short burst on my 1070. Almost wondering if we could parallelize and do two images at a time...

[positlabs]

@bafonso you're right about the CPU bottleneck. It was caused by the run-flownet-many.py script NOT reusing the network, but recreating it for each frame. I have modified the script to use in the docker container. https://github.com/positlabs/fast-artistic-videos-docker/blob/master/run-flownet-many.py

[bafonso]

@bafonso you're right about the CPU bottleneck. It was caused by the run-flownet-many.py script NOT reusing the network, but recreating it for each frame. I have modified the script to use in the docker container. https://github.com/positlabs/fast-artistic-videos-docker/blob/master/run-flownet-many.py

Oh yes, I took your changes and ported it to the dockerize version above and it works much much faster! I have now one core at 100% and my GPU is running anywhere between 50 and 82% so I'm much happier with that! Getting maybe ~3fps on a low res movie right now on my 1070. The docker version was a bit too advanced for my needs this is why I abandoned that idea for now.