search

tensorflow / tensor2tensor

Library of deep learning models and datasets designed to make deep learning more accessible and accelerate ML research.
Apache License 2.0
15.5k stars 3.49k forks source link

absl.flags._exceptions.DuplicateFlagError: The flag 'problem' is defined twice. #1225

Closed stefan-falk closed 5 years ago

stefan-falk commented 5 years ago

I have created my own ASR problem.

However, it does not seem to be possible to use it with t2t-datagen like this:

t2t-datagen --data_dir=/data/problems/my_problem/data --tmp_dir=/data/problems/my_problem/tmp/ --t2t_usr_dir=~/git/asr/t2t_extensions/ --problem=my_problem

This will raise:

  File "/home/sfalk/miniconda3/envs/t2t/lib/python3.5/site-packages/absl/flags/_defines.py", line 241, in DEFINE_string
    DEFINE(parser, name, default, help, flag_values, serializer, **args)
  File "/home/sfalk/miniconda3/envs/t2t/lib/python3.5/site-packages/absl/flags/_defines.py", line 82, in DEFINE
    flag_values, module_name)
  File "/home/sfalk/miniconda3/envs/t2t/lib/python3.5/site-packages/absl/flags/_defines.py", line 104, in DEFINE_flag
    fv[flag.name] = flag
  File "/home/sfalk/miniconda3/envs/t2t/lib/python3.5/site-packages/absl/flags/_flagvalues.py", line 429, in __setitem__
    raise _exceptions.DuplicateFlagError.from_flag(name, self)
absl.flags._exceptions.DuplicateFlagError: The flag 'problem' is defined twice. First from tensor2tensor.bin.t2t_datagen, Second from tensor2tensor.utils.flags.  Description from first occurrence: The name of the problem to generate data for.

and it appears this is coming from this line. Somehow the flags get re-defined. One could just move it into the try-catch block below but I am not sure if this is desired.

stefan-falk commented 5 years ago

Okay so the problem was that I had t2t_trainer imported. I had some testing code 

def main():
  t2t_trainer.main(None)

in the same file where the problem resides. That has not been a problem so far but it seems this does not work if one runs t2t-datagen as the double-definition of certain flags may happen in that case. I simply moved my test code outside my t2t-extensions.