search

capeprivacy / tf-trusted

tf-trusted allows you to run TensorFlow models in secure enclaves
https://capeprivacy.com/
Apache License 2.0
87 stars 11 forks source link

Error while running model #18

Closed kamathhrishi closed 5 years ago

kamathhrishi commented 5 years ago

I followed the instructions to build custom op and began the docker container and got the message

2019-06-06 17:46:22 INFO tf_trusted_enclave.cc : 86 : Server started on port 50051

In a separate command window, I ran the following command 

model_run.py --model_file normal_train2.pb \
               --batch_size 10 \
               --input_file test_input.npy \
               --input_name "image" \
               --output_name "ArgMax"

But, then I ended up receiving the following error.

Traceback (most recent call last): File "/home/hrishikesh/anaconda3/bin/model_run.py", line 7, in <module> exec(compile(f.read(), __file__, 'exec')) File "/home/hrishikesh/tf-trusted/tf_trusted_custom_op/model_run.py", line 27, in <module> model_module = tf.load_op_library(shared_object) File "/home/hrishikesh/anaconda3/lib/python3.7/site-packages/tensorflow/python/framework/load_library.py", line 61, in load_op_library lib_handle = py_tf.TF_LoadLibrary(library_filename) tensorflow.python.framework.errors_impl.NotFoundError: /home/hrishikesh/tf-trusted/tf_trusted_custom_op/model_enclave_op.so: undefined symbol: _ZN10tensorflow12OpDefBuilderC1ESs

Any idea as to why its occurring?

justin1121 commented 5 years ago

This usually occurs when there is an ABI compatibility issue with tensorflow. Can you provide more details about your operating system and versions of python and tensorflow?

kamathhrishi commented 5 years ago

Ubuntu 16.04 , in the above error "/home/hrishikesh/anaconda3/lib/python3.7/site-packages/tensorflow/python/framework/load_library.py" it says 3.7 but then I was using a separate conda environment with 3.5 and tensorflow 1.13.1.

justin1121 commented 5 years ago

Hmm I assume you used the docker container to build the shared object? I'm not sure what else could be wrong. I'd double check and make sure you're actually running 1.13.1 cause thats what the docker container will use.

kamathhrishi commented 5 years ago

@justin1121 If you notice it says "/home/hrishikesh/anaconda3/lib/python3.7/site-packages/tensorflow/python/framework/load_library.py"in the error. When I check python --version it shows 3.5 but conda info shows 3.7.1. Guess its running it using python 3.7?

kamathhrishi commented 5 years ago

Did a couple of things and got it working in a pretty brute forcy manner :)

Thanks for your help.

For some reason, it was running python 3.7 although I had a conda environment of 3.5 which gave the error.

jopasserat commented 5 years ago

As a side note, this only worked for me after installing tensorflow from pip not conda.

That was regardless of the underlying python I created my conda env with (failed with both python3.6 and 3.5).

kamathhrishi commented 5 years ago

Interesting. I did install it with pip too.