converting caffe2 model to Tensorflow pb

[syedaffanhamdani]

I am converting Official ResNet50 model from Caffe2 Model zoo -> NNEF -> Tensorflow. Caffe2 -> NNEF was done using the legacy tool provided which works fine. NNEF to Tensorflow python export works fine as well but conversion tool is throwing the following error while convertion to Tensorflow pb format.

Traceback (most recent call last):
  File "./nnef_tools/convert.py", line 606, in <module>
    convert_using_argv(sys.argv)
  File "./nnef_tools/convert.py", line 594, in convert_using_argv
    conversion_info=args.conversion_info)
  File "./nnef_tools/convert.py", line 426, in convert
    custom_converters=custom_converters))
  File "./nnef_tools/convert.py", line 341, in convert_using_premade_objects
    write_info = writer(target_graph, out_filename)
  File "/home/ubuntu/NNEF-Tools/nnef_tools/io/tensorflow/tf_pb_io.py", line 422, in __call__
    return write_tf_graph_to_protobuf(graph, filename, convert_from_tf_py=self._convert_from_tf_py)
  File "/home/ubuntu/NNEF-Tools/nnef_tools/io/tensorflow/tf_pb_io.py", line 369, in write_tf_graph_to_protobuf
    tf_py_to_tf_pb.convert(graph)
  File "/home/ubuntu/NNEF-Tools/nnef_tools/conversion/tensorflow/tf_py_to_tf_pb.py", line 50, in convert
    assert op.name in _DefaultConverters, "No tf_py_to_tf_pb converter for {}".format(op.name)
AssertionError: No tf_py_to_tf_pb converter for tf.add_n

[gyenesvi]

I believe that the problem may be that the protobuf format does not support the add_n operation (not sure why that is needed for ResNet). I believe it should be work around this with add operation(s). We'll look into this.

[tdanyluk]

I have tried what you did, and confirmed the bug. After that, I have updated the caffe2 converter to produce "add" instead of "add_n" when there are exactly two inputs. Now the conversion fully works.

So, just git pull and do the caffe2 to nnef conversion again, after that the nnef to tensorflow-pb should work.

[syedaffanhamdani]

Thanks, I pulled the new code. Installed it using python setup.py install after which tried to convert. It just halts with following message.

('Set output log: ', 'nnef_output/nnef_convert.log')
Error: Error parsing message

I tried changing the log level to debug but still there is no stack trace..

[tdanyluk]

Sorry to hear that. For me it did work, even after I have reinstalled my NNEF-Tools. I did the following:

pip install protobuf numpy networkx
git clone --recursive https://github.com/KhronosGroup/NNEF-Tools
cd NNEF-Tools/
cd parser/python
python setup.py install
cd ../../legacy/caffe2
mkdir resnet50
cd resnet50
wget https://s3.amazonaws.com/download.caffe2.ai/models/resnet50/predict_net.pb
wget https://s3.amazonaws.com/download.caffe2.ai/models/resnet50/init_net.pb
wget https://raw.githubusercontent.com/caffe2/models/master/resnet50/value_info.json
cd ..
python -m nnef_converter.convert --input-framework caffe2 --output-framework NNEF --input-model resnet50/predict_net.pb  --data-model resnet50/init_net.pb  --value-info resnet50/value_info.json

If you need a stack trace, please edit legacy/caffe2/nnef_converter/convert.py by removing the try-except block at the end of the file. (the main_loop() call should stay there)

Sorry for the inconvenience, please send further details if it still does not work for you.

[tdanyluk]

Also please check if the files that you are supplying to the program are still valid. It should be able to parse.

[syedaffanhamdani]

Hi thanks. I followed your steps, it worked. Have to check accuracy of the new model with MLPerf. I just have two questions:

1. Isn,t it required that Caffe2 should also be installed in the environment?
2. Shouldn't one run python setup.py install in legacy/caffe2 folder as well?

[tdanyluk]

Hi, no problem.

1. It should work without caffe2, because the tool reads the protobuf format directly without the help of caffe2. In general we try to make our converters independent of the framework, you only need the framework for some tests where we actually run the models.
2. You're right, you can run it, but it seems that the tool can be used directly without installing too.

[gyenesvi]

Can this be closed?