search

tensorflow / probability

Probabilistic reasoning and statistical analysis in TensorFlow
https://www.tensorflow.org/probability/
Apache License 2.0
4.24k stars 1.09k forks source link

Module has no attribute error on import #1339

Closed jaanli closed 3 years ago

jaanli commented 3 years ago

This error has been occurring non-deterministically, and seems to interact with the tensorflow datasets pip package:

✖ python                                     
Python 3.9.5 (default, May 18 2021, 19:34:48) 
[GCC 7.3.0] :: Anaconda, Inc. on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import tensorflow_datasets as tfds
_probability.substrates import jax as tfp
2021-05-21 19:18:32.171334: W tensorflow/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libcudart.so.11.0'; dlerror: libcudart.so.11.0: cannot open shared object file: No such file or directory
2021-05-21 19:18:32.171405: I tensorflow/stream_executor/cuda/cudart_stub.cc:29] Ignore above cudart dlerror if you do not have a GPU set up on your machine.
>>> from tensorflow_probability.substrates import jax as tfp
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/jaan/miniconda3/envs/jax/lib/python3.9/site-packages/tensorflow_probability/__init__.py", line 20, in <module>
    from tensorflow_probability import substrates
  File "/home/jaan/miniconda3/envs/jax/lib/python3.9/site-packages/tensorflow_probability/substrates/__init__.py", line 21, in <module>
    from tensorflow_probability.python.internal import all_util
  File "/home/jaan/miniconda3/envs/jax/lib/python3.9/site-packages/tensorflow_probability/python/__init__.py", line 142, in <module>
    dir(globals()[pkg_name])  # Forces loading the package from its lazy loader.
  File "/home/jaan/miniconda3/envs/jax/lib/python3.9/site-packages/tensorflow_probability/python/internal/lazy_loader.py", line 61, in __dir__
    module = self._load()
  File "/home/jaan/miniconda3/envs/jax/lib/python3.9/site-packages/tensorflow_probability/python/internal/lazy_loader.py", line 44, in _load
    module = importlib.import_module(self.__name__)
  File "/home/jaan/miniconda3/envs/jax/lib/python3.9/importlib/__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "/home/jaan/miniconda3/envs/jax/lib/python3.9/site-packages/tensorflow_probability/python/experimental/__init__.py", line 35, in <module>
    from tensorflow_probability.python.experimental import bijectors
  File "/home/jaan/miniconda3/envs/jax/lib/python3.9/site-packages/tensorflow_probability/python/experimental/bijectors/__init__.py", line 19, in <module>
    from tensorflow_probability.python.experimental.bijectors.distribution_bijectors import make_distribution_bijector
  File "/home/jaan/miniconda3/envs/jax/lib/python3.9/site-packages/tensorflow_probability/python/experimental/bijectors/distribution_bijectors.py", line 25, in <module>
    from tensorflow_probability.python.distributions import deterministic
  File "/home/jaan/miniconda3/envs/jax/lib/python3.9/site-packages/tensorflow_probability/python/distributions/__init__.py", line 109, in <module>
    from tensorflow_probability.python.distributions.pixel_cnn import PixelCNN
  File "/home/jaan/miniconda3/envs/jax/lib/python3.9/site-packages/tensorflow_probability/python/distributions/pixel_cnn.py", line 37, in <module>
    from tensorflow_probability.python.layers import weight_norm
  File "/home/jaan/miniconda3/envs/jax/lib/python3.9/site-packages/tensorflow_probability/python/layers/__init__.py", line 31, in <module>
    from tensorflow_probability.python.layers.distribution_layer import CategoricalMixtureOfOneHotCategorical
  File "/home/jaan/miniconda3/envs/jax/lib/python3.9/site-packages/tensorflow_probability/python/layers/distribution_layer.py", line 72, in <module>
    tf.keras.__internal__.utils.register_symbolic_tensor_type(dtc._TensorCoercible)  # pylint: disable=protected-access
AttributeError: module 'tensorflow.python.keras.api._v2.keras' has no attribute '__internal__'
>>>

Anaconda environment.yml file on Ubuntu 20.04: https://github.com/altosaar/variational-autoencoder/blob/master/environment_jax.yml

I just did a clean reinstall in this environment using:

pip install --upgrade pip
pip install tensorflow
pip install tensorflow-datasets
pip install --upgrade jax jaxlib  # CPU-only version
pip install tfp-nightly

Any ideas on how to debug this? Thanks so much!

brianwa84 commented 3 years ago

Tfp nightly works with tf nightly. We just put up 0.13.0rc0 for tf 2.5.0, can you confirm that works ok?

On Fri, May 21, 2021, 7:23 PM Jaan Altosaar @.***> wrote:

This error has been occurring non-deterministically, and seems to interact with the tensorflow datasets pip package:

✖ python

Python 3.9.5 (default, May 18 2021, 19:34:48)

[GCC 7.3.0] :: Anaconda, Inc. on linux

Type "help", "copyright", "credits" or "license" for more information.

import tensorflow_datasets as tfds

_probability.substrates import jax as tfp

2021-05-21 19:18:32.171334: W tensorflow/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libcudart.so.11.0'; dlerror: libcudart.so.11.0: cannot open shared object file: No such file or directory

2021-05-21 19:18:32.171405: I tensorflow/stream_executor/cuda/cudart_stub.cc:29] Ignore above cudart dlerror if you do not have a GPU set up on your machine.

from tensorflow_probability.substrates import jax as tfp

Traceback (most recent call last):

File "", line 1, in

File "/home/jaan/miniconda3/envs/jax/lib/python3.9/site-packages/tensorflow_probability/init.py", line 20, in 

from tensorflow_probability import substrates

File "/home/jaan/miniconda3/envs/jax/lib/python3.9/site-packages/tensorflow_probability/substrates/init.py", line 21, in 

from tensorflow_probability.python.internal import all_util

File "/home/jaan/miniconda3/envs/jax/lib/python3.9/site-packages/tensorflow_probability/python/init.py", line 142, in 

dir(globals()[pkg_name])  # Forces loading the package from its lazy loader.

File "/home/jaan/miniconda3/envs/jax/lib/python3.9/site-packages/tensorflow_probability/python/internal/lazy_loader.py", line 61, in dir

module = self._load()

File "/home/jaan/miniconda3/envs/jax/lib/python3.9/site-packages/tensorflow_probability/python/internal/lazy_loader.py", line 44, in _load

module = importlib.import_module(self.__name__)

File "/home/jaan/miniconda3/envs/jax/lib/python3.9/importlib/init.py", line 127, in import_module

return _bootstrap._gcd_import(name[level:], package, level)

File "/home/jaan/miniconda3/envs/jax/lib/python3.9/site-packages/tensorflow_probability/python/experimental/init.py", line 35, in 

from tensorflow_probability.python.experimental import bijectors

File "/home/jaan/miniconda3/envs/jax/lib/python3.9/site-packages/tensorflow_probability/python/experimental/bijectors/init.py", line 19, in 

from tensorflow_probability.python.experimental.bijectors.distribution_bijectors import make_distribution_bijector

File "/home/jaan/miniconda3/envs/jax/lib/python3.9/site-packages/tensorflow_probability/python/experimental/bijectors/distribution_bijectors.py", line 25, in 

from tensorflow_probability.python.distributions import deterministic

File "/home/jaan/miniconda3/envs/jax/lib/python3.9/site-packages/tensorflow_probability/python/distributions/init.py", line 109, in 

from tensorflow_probability.python.distributions.pixel_cnn import PixelCNN

File "/home/jaan/miniconda3/envs/jax/lib/python3.9/site-packages/tensorflow_probability/python/distributions/pixel_cnn.py", line 37, in 

from tensorflow_probability.python.layers import weight_norm

File "/home/jaan/miniconda3/envs/jax/lib/python3.9/site-packages/tensorflow_probability/python/layers/init.py", line 31, in 

from tensorflow_probability.python.layers.distribution_layer import CategoricalMixtureOfOneHotCategorical

File "/home/jaan/miniconda3/envs/jax/lib/python3.9/site-packages/tensorflow_probability/python/layers/distribution_layer.py", line 72, in 

tf.keras.__internal__.utils.register_symbolic_tensor_type(dtc._TensorCoercible)  # pylint: disable=protected-access

AttributeError: module 'tensorflow.python.keras.api._v2.keras' has no attribute 'internal'

Anaconda environment.yml file on Ubuntu 20.04: https://github.com/altosaar/variational-autoencoder/blob/master/environment_jax.yml

I just did a clean reinstall in this environment using:

pip install --upgrade pip

pip install tensorflow

pip install tensorflow-datasets

pip install --upgrade jax jaxlib # CPU-only version

pip install tfp-nightly

Any ideas on how to debug this? Thanks so much!

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/tensorflow/probability/issues/1339, or unsubscribe https://github.com/notifications/unsubscribe-auth/AFJFSI2ZFPKMUH5I5EZNVKDTO3TOBANCNFSM45KAHPXQ .

jaanli commented 3 years ago

Got it, that was my mistake. Thank you for the pointer!

I didn't know that tfp-nightly may have changes that break compatibility with tf-nightly.

Fingers crossed a stable release comes soon that fixes the compatibility issues with jax that make first-time use difficult: https://github.com/google/jax/issues/6801