Failed to import test module: tapas.experiments.prediction_utils_test when using tox on Python 3.7

[GaurangTandon]

Hi, so, rerunning the repo with Python 3.7 and finally on step tox, got this error (I am skipping the warning outputs that came before this):

======================================================================
ERROR: tapas.experiments.prediction_utils_test (unittest.loader._FailedTest)
----------------------------------------------------------------------
ImportError: Failed to import test module: tapas.experiments.prediction_utils_test
Traceback (most recent call last):
  File "/usr/local/apps/python-3.7.4/lib/python3.7/unittest/loader.py", line 436, in _find_test_path
    module = self._get_module_from_name(name)
  File "/usr/local/apps/python-3.7.4/lib/python3.7/unittest/loader.py", line 377, in _get_module_from_name
    __import__(name)
  File "/home/gaurang.tandon/tables/tapas/tapas/experiments/prediction_utils_test.py", line 21, in <module>
    from tapas.experiments import prediction_utils
  File "/home/gaurang.tandon/tables/tapas/tapas/experiments/prediction_utils.py", line 25, in <module>
    from tapas.models import tapas_classifier_model
  File "/home/gaurang.tandon/tables/tapas/tapas/models/tapas_classifier_model.py", line 31, in <module>
    import tensorflow_probability as tfp
  File "/home/gaurang.tandon/tables/tapas/.tox/py37/lib/python3.7/site-packages/tensorflow_probability/__init__.py", line 76, in <module>
    from tensorflow_probability.python import *  # pylint: disable=wildcard-import
  File "/home/gaurang.tandon/tables/tapas/.tox/py37/lib/python3.7/site-packages/tensorflow_probability/python/__init__.py", line 23, in <module>
    from tensorflow_probability.python import distributions
  File "/home/gaurang.tandon/tables/tapas/.tox/py37/lib/python3.7/site-packages/tensorflow_probability/python/distributions/__init__.py", line 88, in <module>
    from tensorflow_probability.python.distributions.pixel_cnn import PixelCNN
  File "/home/gaurang.tandon/tables/tapas/.tox/py37/lib/python3.7/site-packages/tensorflow_probability/python/distributions/pixel_cnn.py", line 37, in <module>
    from tensorflow_probability.python.layers import weight_norm
  File "/home/gaurang.tandon/tables/tapas/.tox/py37/lib/python3.7/site-packages/tensorflow_probability/python/layers/__init__.py", line 31, in <module>
    from tensorflow_probability.python.layers.distribution_layer import CategoricalMixtureOfOneHotCategorical
  File "/home/gaurang.tandon/tables/tapas/.tox/py37/lib/python3.7/site-packages/tensorflow_probability/python/layers/distribution_layer.py", line 28, in <module>
    from cloudpickle.cloudpickle import CloudPickler
ImportError: cannot import name 'CloudPickler' from 'cloudpickle.cloudpickle' (/home/gaurang.tandon/tables/tapas/.tox/py37/lib/python3.7/site-packages/cloudpickle/cloudpickle.py)

======================================================================
ERROR: tapas.models.tapas_classifier_model_test (unittest.loader._FailedTest)
----------------------------------------------------------------------
ImportError: Failed to import test module: tapas.models.tapas_classifier_model_test
Traceback (most recent call last):
  File "/usr/local/apps/python-3.7.4/lib/python3.7/unittest/loader.py", line 436, in _find_test_path
    module = self._get_module_from_name(name)
  File "/usr/local/apps/python-3.7.4/lib/python3.7/unittest/loader.py", line 377, in _get_module_from_name
    __import__(name)
  File "/home/gaurang.tandon/tables/tapas/tapas/models/tapas_classifier_model_test.py", line 22, in <module>
    from tapas.models import tapas_classifier_model
  File "/home/gaurang.tandon/tables/tapas/tapas/models/tapas_classifier_model.py", line 31, in <module>
    import tensorflow_probability as tfp
  File "/home/gaurang.tandon/tables/tapas/.tox/py37/lib/python3.7/site-packages/tensorflow_probability/__init__.py", line 76, in <module>
    from tensorflow_probability.python import *  # pylint: disable=wildcard-import
  File "/home/gaurang.tandon/tables/tapas/.tox/py37/lib/python3.7/site-packages/tensorflow_probability/python/__init__.py", line 23, in <module>
    from tensorflow_probability.python import distributions
  File "/home/gaurang.tandon/tables/tapas/.tox/py37/lib/python3.7/site-packages/tensorflow_probability/python/distributions/__init__.py", line 88, in <module>
    from tensorflow_probability.python.distributions.pixel_cnn import PixelCNN
  File "/home/gaurang.tandon/tables/tapas/.tox/py37/lib/python3.7/site-packages/tensorflow_probability/python/distributions/pixel_cnn.py", line 37, in <module>
    from tensorflow_probability.python.layers import weight_norm
  File "/home/gaurang.tandon/tables/tapas/.tox/py37/lib/python3.7/site-packages/tensorflow_probability/python/layers/__init__.py", line 31, in <module>
    from tensorflow_probability.python.layers.distribution_layer import CategoricalMixtureOfOneHotCategorical
  File "/home/gaurang.tandon/tables/tapas/.tox/py37/lib/python3.7/site-packages/tensorflow_probability/python/layers/distribution_layer.py", line 28, in <module>
    from cloudpickle.cloudpickle import CloudPickler
ImportError: cannot import name 'CloudPickler' from 'cloudpickle.cloudpickle' (/home/gaurang.tandon/tables/tapas/.tox/py37/lib/python3.7/site-packages/cloudpickle/cloudpickle.py)

----------------------------------------------------------------------
Ran 163 tests in 21.259s

FAILED (errors=2, skipped=6)
ERROR: InvocationError for command /home/gaurang.tandon/tables/tapas/.tox/py37/bin/python -m unittest discover -p '*_test.py' (exited with code 1)
_____________________________________________________ summary _____________________________________________________
ERROR:  py36: InterpreterNotFound: python3.6
ERROR:   py37: commands failed

I had completed python -m pip install -e . succesfully. What should I do here?

[muelletm]

I don't know what's going on there but it's not happening in our code. It looks like tensorflow_probability or at least cloudpickle wasn't installed correctly. Maybe you can understand why if you go through the warning logs.

[eisenjulian]

Apparently there was an issue with a tensorflow probability dependency, and the right fix at the moment is changing the version dependency to 0.10.1 as mentioned in https://github.com/tensorflow/probability/issues/991 We will release a new version soon, in the meantime you can update the requirements file directly.

[ghost]

Closing for now ...