Trying to train translate_ende_wmt32k demo with ml-engine, got tensorflow version mismatch error.
looks like t2t 1.11 introduced tensorflow_probability, which added an dependency of tensorflow 1.12. Based on the ML Engine runtime versions: it currently supports up to tensorflow 1.10. Thus t2t 1.11 can no longer submit ML Engine training job.
Environment information
OS: <your answer here>
$ pip freeze | grep tensor
# your output here
mesh-tensorflow==0.0.4
tensor2tensor==1.11.0
tensorboard==1.12.0
tensorflow==1.12.0
tensorflow-hub==0.1.1
tensorflow-metadata==0.9.0
tensorflow-probability==0.5.0
$ python -V
# your output here
Python 3.6.6
# Error logs: From ML Engine log:
Traceback (most recent call last):
File "/usr/lib/python3.5/runpy.py", line 184, in _run_module_as_main
"__main__", mod_spec)
File "/usr/lib/python3.5/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "/root/.local/lib/python3.5/site-packages/tensor2tensor/bin/t2t_trainer.py", line 25, in <module>
from tensor2tensor import models # pylint: disable=unused-import
File "/root/.local/lib/python3.5/site-packages/tensor2tensor/models/__init__.py", line 22, in <module>
from tensor2tensor.layers import modalities # pylint: disable=g-import-not-at-top
File "/root/.local/lib/python3.5/site-packages/tensor2tensor/layers/modalities.py", line 22, in <module>
from tensor2tensor.layers import common_attention
File "/root/.local/lib/python3.5/site-packages/tensor2tensor/layers/common_attention.py", line 31, in <module>
from tensor2tensor.layers import common_layers
File "/root/.local/lib/python3.5/site-packages/tensor2tensor/layers/common_layers.py", line 32, in <module>
import tensorflow_probability as tfp
File "/root/.local/lib/python3.5/site-packages/tensorflow_probability/__init__.py", line 68, in <module>
_ensure_tf_install()
File "/root/.local/lib/python3.5/site-packages/tensorflow_probability/__init__.py", line 65, in _ensure_tf_install
present=tf.__version__))
ImportError: This version of TensorFlow Probability requires TensorFlow version >= 1.12.0; Detected an installation of version 1.9.0. Please upgrade TensorFlow to proceed.
dcompgriff
commented
5 years ago
Description
Trying to train translate_ende_wmt32k demo with ml-engine, got tensorflow version mismatch error.
looks like t2t 1.11 introduced tensorflow_probability, which added an dependency of tensorflow 1.12. Based on the ML Engine runtime versions: it currently supports up to tensorflow 1.10. Thus t2t 1.11 can no longer submit ML Engine training job.
Environment information
For bugs: reproduction and error logs
just follow ML Engine example