tf-hub falling back to CPU

[VinuraD]

Hi,

I'm using LaBSE model from the tf-hub and running on tensorflow-gpu (v2.2.0) on conda virtual environment. I've checked whether the tensorflow-gpu version is running correctly and have run other models (Huggingface) too. However, I observed below output when installing pip packages related to tf-hub model. Python=3.8 CUDA=10.1

!pip install tensorflow-text
!pip install tf-models-official

Requirement already satisfied: tensorflow-text in /userdirs/test/miniconda3/envs/tensorflowenv/lib/python3.8/site-packages (2.5.0)
Requirement already satisfied: tensorflow-hub>=0.8.0 in /userdirs/test/miniconda3/envs/tensorflowenv/lib/python3.8/site-packages (from tensorflow-text) (0.12.0)
Collecting tensorflow<2.6,>=2.5.0
  Using cached tensorflow-2.5.0-cp38-cp38-manylinux2010_x86_64.whl (454.4 MB)
....
Installing collected packages: tensorflow, tensorflow-text
  Attempting uninstall: tensorflow
    Found existing installation: tensorflow 2.2.0
    Uninstalling tensorflow-2.2.0:
      Successfully uninstalled tensorflow-2.2.0
Successfully installed tensorflow-2.5.0 tensorflow-text-2.5.0

After that, the tensorflow version falls to v.2.5.0 (CPU) and runs. I tried manually removing this tensorflow version but it happens to install again. If I run it without tensorflow 2.5.0,(skipping above two pip install steps and using tf-gpu 2.2.0) it gives an error,

ModuleNotFoundError: No module named 'tensorflow.python.types'

Is there some way that I could install tf-hub and related libraries on GPU or is it not compatible with the tensorflow version I'm using?. I did not find a specific instruction here; https://www.tensorflow.org/hub/installation

import tensorflow_hub as hub
import tensorflow as tf
#import tensorflow_text as text  # 
import numpy as np
#from official.nlp import optimization  #

ModuleNotFoundError                       Traceback (most recent call last)
<ipython-input-9-1be3e694b708> in <module>
----> 1 import tensorflow_hub as hub
      2 import tensorflow as tf
      3 import tensorflow_text as text  # Needed for loading universal-sentence-encoder-cmlm/multilingual-preprocess
      4 import numpy as np
      5 from official.nlp import optimization  # to create AdamW optimizer

~/miniconda3/envs/tensorflowenv/lib/python3.8/site-packages/tensorflow_hub/__init__.py in <module>
     86 
     87 
---> 88 from tensorflow_hub.estimator import LatestModuleExporter
     89 from tensorflow_hub.estimator import register_module_for_export
     90 from tensorflow_hub.feature_column import image_embedding_column

~/miniconda3/envs/tensorflowenv/lib/python3.8/site-packages/tensorflow_hub/estimator.py in <module>
     60 
     61 
---> 62 class LatestModuleExporter(tf.compat.v1.estimator.Exporter):
     63   """Regularly exports registered modules into timestamped directories.
     64 

~/miniconda3/envs/tensorflowenv/lib/python3.8/site-packages/tensorflow/python/util/lazy_loader.py in __getattr__(self, item)
     60 
     61   def __getattr__(self, item):
---> 62     module = self._load()
     63     return getattr(module, item)
     64 

~/miniconda3/envs/tensorflowenv/lib/python3.8/site-packages/tensorflow/python/util/lazy_loader.py in _load(self)
     43     """Load the module and insert it into the parent's globals."""
     44     # Import the target module and insert it into the parent's namespace
---> 45     module = importlib.import_module(self.__name__)
     46     self._parent_module_globals[self._local_name] = module
     47 

~/miniconda3/envs/tensorflowenv/lib/python3.8/importlib/__init__.py in import_module(name, package)
    125                 break
    126             level += 1
--> 127     return _bootstrap._gcd_import(name[level:], package, level)
    128 
    129 

~/miniconda3/envs/tensorflowenv/lib/python3.8/site-packages/tensorflow_estimator/python/estimator/api/_v1/estimator/__init__.py in <module>
      8 import sys as _sys
      9 
---> 10 from tensorflow_estimator.python.estimator.api._v1.estimator import experimental
     11 from tensorflow_estimator.python.estimator.api._v1.estimator import export
     12 from tensorflow_estimator.python.estimator.api._v1.estimator import inputs

~/miniconda3/envs/tensorflowenv/lib/python3.8/site-packages/tensorflow_estimator/python/estimator/api/_v1/estimator/experimental/__init__.py in <module>
      8 import sys as _sys
      9 
---> 10 from tensorflow_estimator.python.estimator.canned.dnn import dnn_logit_fn_builder
     11 from tensorflow_estimator.python.estimator.canned.kmeans import KMeansClustering as KMeans
     12 from tensorflow_estimator.python.estimator.canned.linear import LinearSDCA

~/miniconda3/envs/tensorflowenv/lib/python3.8/site-packages/tensorflow_estimator/python/estimator/canned/dnn.py in <module>
     29 from tensorflow.python.keras.utils import losses_utils
     30 from tensorflow.python.util.tf_export import estimator_export
---> 31 from tensorflow_estimator.python.estimator import estimator
     32 from tensorflow_estimator.python.estimator.canned import head as head_lib
     33 from tensorflow_estimator.python.estimator.canned import optimizers

~/miniconda3/envs/tensorflowenv/lib/python3.8/site-packages/tensorflow_estimator/python/estimator/estimator.py in <module>
     49 from tensorflow.python.util import tf_contextlib
     50 from tensorflow.python.util.tf_export import estimator_export
---> 51 from tensorflow_estimator.python.estimator import model_fn as model_fn_lib
     52 from tensorflow_estimator.python.estimator import run_config
     53 from tensorflow_estimator.python.estimator import util as estimator_util

~/miniconda3/envs/tensorflowenv/lib/python3.8/site-packages/tensorflow_estimator/python/estimator/model_fn.py in <module>
     27 from tensorflow.python.saved_model import model_utils as export_utils
     28 from tensorflow.python.tpu import tensor_tracer
---> 29 from tensorflow.python.types import core
     30 from tensorflow.python.util import function_utils
     31 from tensorflow.python.util.tf_export import estimator_export

ModuleNotFoundError: No module named 'tensorflow.python.types'

[arghyaganguly]

@VinuraD , thanks for reporting this issue. I tried reproducing this issue with the steps and versions mentioned in your comment and faced same errors. I've also tried with a conda environment using :

a)tensorflow-gpu==2.2.1 b)tensorflow-text==2.2.0 c)tensorflow-hub==0.12.0 d)python 3.8.10

by following the below steps:-

a)pip install tensorflow-gpu==2.2.1 b)pip install tensorflow-text==2.2.0 installs tensorflow==2.2.3 (doesn't uninstall tensorflow-gpu==2.2.1) c)pip install tensorflow-hub==0.12.0

after this when i import tensorflow,tensorflow_text and tensorflow_hub using import statements in a notebook with the conda kernel ,i donot face any error.The tensorflow version is also being shown as 2.2.1 (gpu). Also, i've set my cuda device in the notebook by :- !export CUDA_VISIBLE_DEVICES=0

[VinuraD]

Hi, I later tried with tensorflow-gpu v.2.4.1 (which is shown as the next available & compatible version in conda search). and used (Python 3.7, CUDA 10.1)

pip install tensorflow-text==2.4.1
pip install tf-models-official==2.4.0

then I get an error simiar to https://github.com/tensorflow/text/issues/385, except there gpu is not used.

[MorganR]

Hi @VinuraD, can you please provide a code sample that demonstrates where the error happens, and the original error text?

Generally speaking, it sounds like you are running into a compatibility issue between the versions of tensorflow-text and tensorflow that you are using. It sounds like the version requirements that tensorflow-text imposes on tensorflow are quite strict. I suspect the issue might be that you are using the gpu version of tensorflow, instead of the cpu version.

How are you installing tensorflow? Based on previous questions in the TF Text repo (example), the recommended approach is to install your tensorflow version after you install tensorflow-text.

[VinuraD]

@MorganR The error text related to my original issue is mentioned in my first comment. The error text related to the 2nd comment is here https://github.com/tensorflow/text/issues/644 (which I opened separately). Will this be sufficient?

I am installing tensorflow-gpu first (before installing the tensorflow-text), using conda install -c anaconda tensorflow-gpu==2.4.1 in a conda virtual env, which is a setup working with other gpu related runs. Only this came up with errors.

The CPU is enough just to get the embeddings. But it takes a lot of time for fine tuning the model with added layers only with CPU.

[MorganR]

Thanks for clarifying and for opening tensorflow/text#644. Since all these issues are related to compatibility between your tensorflow-text and tensorflow versions, and are not related to TF Hub, I am closing this issue.