search

tensorflow / hub

A library for transfer learning by reusing parts of TensorFlow models.
https://tensorflow.org/hub
Apache License 2.0
3.49k stars 1.67k forks source link

Multilingual model - tf_sentencepiece symbol errors #339

Closed chewbecca-80 closed 5 years ago

chewbecca-80 commented 5 years ago

When trying to replicate the Universal Sentence Encoder Multilingual tool example (https://colab.research.google.com/github/tensorflow/hub/blob/master/examples/colab/cross_lingual_similarity_with_tf_hub_multilingual_universal_encoder.ipynb) in a conda and virtual environment, symbol errors arise when importing tf_sentencepiece. I have attached several conda environment yml files to this ticket, all of which produce the same error when importing tf_sentencepiece. use_ml_environments.zip

To reproduce the error:

  1. Create a conda environment using one of the environment yml files with the command: conda env create -f filename where filename is the name of file.
  2. Activate the conda environment conda activate environment-name where environment-name is the name of the environment (you can see the environment name after creating it, or seeing a list of environments with the command "conda info --envs")
  3. Open a python shell python
  4. Import tensorflow import tensorflow as tf
  5. Import tf_sentencepiece import tf_sentencepiece
  6. View the error

Traceback (most recent call last): File "", line 1, in File "/home/rservaites/miniconda/envs/tf-test-gpu-1-14/lib/python3.7/site-packages/tf_sentencepiece/init.py", line 5, in from tf_sentencepiece.sentencepiece_processor_ops import * File "/home/rservaites/miniconda/envs/tf-test-gpu-1-14/lib/python3.7/site-packages/tf_sentencepiece/sentencepiece_processor_ops.py", line 43, in _gen_sentencepiece_processor_op = tf.load_op_library(so_file) File "/home/rservaites/miniconda/envs/tf-test-gpu-1-14/lib/python3.7/site-packages/tensorflow/python/framework/load_library.py", line 61, in load_op_library lib_handle = py_tf.TF_LoadLibrary(library_filename) tensorflow.python.framework.errors_impl.NotFoundError: /home/rservaites/miniconda/envs/tf-test-gpu-1-14/lib/python3.7/site-packages/tf_sentencepiece/_sentencepiece_processor_ops.so.1.14.0: undefined symbol: _ZN10tensorflow12OpDefBuilder4AttrESs

gowthamkpr commented 5 years ago

Try downgrading python version to 3.6 and let me know if you still face the same error. Thanks!

chewbecca-80 commented 5 years ago

HI! I originally tried everything with python 3.6 and it came back with the same errors.

gowthamkpr commented 5 years ago

@chewbecca-80 The issue might be that you might have installed tensorflow using conda and tf_sentencepiece using pip. This causes incompatibility.

Also please note that this issue is not related to tensorflow Hub. Its mainly related to google/sentencepiece. For further clarifications and to get fast response please post your issue here.

chewbecca-80 commented 5 years ago

Unfortunately, that's not the issue. I originally installed tensorflow using pip by following the example given in the Universal Sentence Encoder Multilingual Colab notebook. When I installed it that way, I got an error when I tried to import tensorflow. But you're right, I should make this issue in sentencepiece. I just clicked the link in the USEML documentation and it brought me to here for reporting issues. Thanks!! :)