search

tensorflow / tensor2tensor

Library of deep learning models and datasets designed to make deep learning more accessible and accelerate ML research.
Apache License 2.0
15.18k stars 3.45k forks source link

Training translate_ende_wmt32k with reinforcement learning ? #1352

Open yashkumaratri opened 5 years ago

yashkumaratri commented 5 years ago

Description

I am jus giving this repo a try , So as per the docs I have trained this probelm=translate_ende_wmt32k stated in tensor2tensor Basics walkthrough and Now I want to tune themodel using reinforcement learning using model based training. So how to move forward as no docs points to continue this tutorial ...

Environment information

OS: Ubuntu 16.04

$ pip freeze | grep tensor
# your output here
tensor2tensor==1.11.0
tensorboard==1.10.0
tensorflow-gpu==1.12.0
tensorflow-metadata==0.9.0
tensorflow-probability==0.5.0
tensorflow-tensorboard==0.4.0

$ python -V
# your output here
Python 3.5.2

For bugs: reproduction and error logs

# Steps to reproduce:
...
# Error logs:
...

python3 -m tensor2tensor.rl.trainer_model_based   --output_dir=$OUT_DIR   --loop_hparams_set=rl_modelrl_base   --loop_hparams='game=freeway'

Traceback (most recent call last):
  File "/usr/lib/python3.5/runpy.py", line 184, in _run_module_as_main
    "__main__", mod_spec)
  File "/usr/lib/python3.5/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/home/paperspace/.local/lib/python3.5/site-packages/tensor2tensor/rl/trainer_model_based.py", line 604, in <module>
    tf.app.run()
  File "/home/paperspace/.local/lib/python3.5/site-packages/tensorflow/python/platform/app.py", line 125, in run
    _sys.exit(main(argv))
  File "/home/paperspace/.local/lib/python3.5/site-packages/tensor2tensor/rl/trainer_model_based.py", line 597, in main
    hp = trainer_model_based_params.create_loop_hparams()
  File "/home/paperspace/.local/lib/python3.5/site-packages/tensor2tensor/rl/trainer_model_based_params.py", line 834, in create_loop_hparams
    hparams = registry.hparams(FLAGS.loop_hparams_set)
  File "/home/paperspace/.local/lib/python3.5/site-packages/tensor2tensor/utils/registry.py", line 157, in hparams
    display_list_by_prefix(list_hparams(), starting_spaces=4)))
LookupError: HParams set rl_modelrl_base never registered. Sets registered:
    adaptive:
      * adaptive_universal_transformer_base
      * adaptive_universal_transformer_base_dropout03
      * adaptive_universal_transformer_base_dropout05
      * adaptive_universal_transformer_concat_tiny
      * adaptive_universal_transformer_global_base
      * adaptive_universal_transformer_mix_after_ut_base
      * adaptive_universal_transformer_mix_before_ut_base
      * adaptive_universal_transformer_position_random_timing_tiny
.....................................................................................
zhangzhen999 commented 5 years ago

Please let me see the file of your reinforcement learning model. Maybe I can help you, and I have encountered similar problems