t2t_usr_dir not working in polyaxon container

flackbash commented 5 years ago

Description

I'm trying to use t2t-datagen to generate my own data following the example here. I'm running the program in a polyaxon container However, I'm getting the error

ValueError: You must specify one of the supported problems to generate data for:
* algorithmic_addition_binary40
* algorithmic_addition_decimal40
...

When I'm running the same code on my local machine it's working without errors.

Environment information (Container)

OS: Ubuntu 16.04.5 LTS

$ pip freeze | grep tensor
mesh-tensorflow==0.0.5
tensor2tensor==1.13.2
tensorboard==1.13.0
tensorflow-datasets==1.0.1
tensorflow-estimator==1.13.0
tensorflow-gpu==1.13.1
tensorflow-metadata==0.13.0
tensorflow-probability==0.6.0

$ python -V
Python 3.5.2

For bugs: reproduction and error logs

I have a directory /code/ that contains both, a file named poetry_lines.py that starts with:

from tensor2tensor.data_generators import problem
from tensor2tensor.data_generators import text_problems
from tensor2tensor.utils import registry

@registry.register_problem
class PoetryLines(text_problems.Text2TextProblem):
...

and a file __init__.py that imports poetry_lines with:

from . import poetry_lines

When I call

/usr/local/bin/t2t-datagen --t2t_usr_dir=/code/ --data_dir=data/ --tmp_dir=/tmp/t2t_datagen/ --problem=poetry_lines

I get the following output

INFO:tensorflow:Importing user module code from path /
2019-04-11T15:29:44.046964953Z
WARNING: The TensorFlow contrib module will not be included in TensorFlow 2.0.
For more information, please see:
* https://github.com/tensorflow/community/blob/master/rfcs/20180907-contrib-sunset.md
* https://github.com/tensorflow/addons
If you depend on functionality not listed there, please file an issue.
2019-04-11T15:29:44.047024927Z
Traceback (most recent call last):
File "/usr/local/bin/t2t-datagen", line 28, in <module>
tf.app.run()
File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/platform/app.py", line 125, in run
_sys.exit(main(argv))
File "/usr/local/bin/t2t-datagen", line 23, in main
t2t_datagen.main(argv)
File "/usr/local/lib/python3.5/dist-packages/tensor2tensor/bin/t2t_datagen.py", line 196, in main
raise ValueError(error_msg)
ValueError: You must specify one of the supported problems to generate data for:
* algorithmic_addition_binary40
* algorithmic_addition_decimal40
* algorithmic_algebra_inverse
* algorithmic_cipher_shift200
* algorithmic_cipher_shift5
* algorithmic_cipher_vigenere200
...

svenster16 commented 5 years ago

I have followed steps very similar to yours and I have gotten the same error. When I run it locally on my laptop I can register the problem, but when I run it on this docker container I get the error.

aswin-giridhar commented 4 years ago

@flackbash and @svenster16 Where you guys able to solve this issue?

flackbash commented 4 years ago

I was not unfortunately :/

aswin-giridhar commented 4 years ago

@flackbash thanks for the quick reply.

tensorflow / tensor2tensor