t2t-decoder: LookupError: unknown encoding

[123mitnik]

Description

Successfully trained a model with the provided timeseries toy example problem tensor2tensor/tensor2tensor/data_generators/timeseries.py

after running the t2t-decoder it barked with (full traceback below) :

LookupError: unknown encoding

started the t2t-decoder with all the standard params

BEAM_SIZE=4 ALPHA=0.6

t2t-decoder \ --t2t_usr_dir=/Users/sampleUser1/myExperiment/t2t \ --data_dir=$DATA_DIR \ --problem=$PROBLEM \ --model=$MODEL \ --hparams_set=$HPARAMS \ --output_dir=$TRAIN_DIR \ --decode_hparams="beam_size=$BEAM_SIZE,alpha=$ALPHA" \ --decode_from_file=/predictThis.csv \ --decode_to_file=/predicted.csv ...

Environment information

OS: masOS 10.13.4

$ pip freeze | grep tensor
-e git+https://github.com/tensorflow/tensor2tensor.git@6969fab42200a7da11bc40c9537b76b0a204b46a#egg=tensor2tensor
tensorboard==1.8.0
tensorflow==1.8.0
tensorflow-tensorboard==0.4.0rc2

$ python -V
Python 2.7.14 :: Anaconda custom (64-bit)

### For bugs: reproduction and error logs

`Traceback (most recent call last):
  File "/anaconda2/bin/t2t-decoder", line 6, in <module>
    exec(compile(open(__file__).read(), __file__, 'exec'))
  File "/Downloads/tensor2tensor/tensor2tensor/bin/t2t-decoder", line 16, in <module>
    tf.app.run()
  File "/anaconda2/lib/python2.7/site-packages/tensorflow/python/platform/app.py", line 126, in run
    _sys.exit(main(argv))
  File "/Downloads/tensor2tensor/tensor2tensor/bin/t2t-decoder", line 12, in main
    t2t_decoder.main(argv)
  File "/Downloads/tensor2tensor/tensor2tensor/bin/t2t_decoder.py", line 188, in main
    decode(estimator, hp, decode_hp)
  File "/Downloads/tensor2tensor/tensor2tensor/bin/t2t_decoder.py", line 91, in decode
    checkpoint_path=FLAGS.checkpoint_path)
  File "/Downloads/tensor2tensor/tensor2tensor/utils/decoding.py", line 261, in decode_from_file
    for result in result_iter:
  File "/anaconda2/lib/python2.7/site-packages/tensorflow/python/estimator/estimator.py", line 494, in predict
    input_fn, model_fn_lib.ModeKeys.PREDICT)
  File "/anaconda2/lib/python2.7/site-packages/tensorflow/python/estimator/estimator.py", line 670, in _get_features_from_input_fn
    result = self._call_input_fn(input_fn, mode)
  File "/anaconda2/lib/python2.7/site-packages/tensorflow/python/estimator/estimator.py", line 798, in _call_input_fn
    return input_fn(**kwargs)
  File "/Downloads/tensor2tensor/tensor2tensor/utils/decoding.py", line 255, in input_fn
    gen_fn = make_input_fn_from_generator(input_gen)
  File "/Downloads/tensor2tensor/tensor2tensor/utils/decoding.py", line 320, in make_input_fn_from_generator
    first_ex = six.next(gen)
  File "/Downloads/tensor2tensor/tensor2tensor/utils/decoding.py", line 389, in _decode_batch_input_fn
    input_ids = vocabulary.encode(inputs)
LookupError: unknown encoding: -2.1981273389651914 1.4754402241238949 -2.0364070394907143 0.006102478719054525 -0.005413125753461879 -1.1473868167510746 -1.5742127274333766 -0.8920022684498778 -0.9699055127239368 -1.5503521288926254 -1.8973860674703469 -1.8973860674703469 -1.8973860674703469 -1.8973860674703469 -2.1981273389651914 -0.3019231673748857 1.4504621765305052 1.3931785178761864 1.5240167041767074 1.6484017478940642 1.1835589237247548 0.8656818492627395 1.131335753674254 1.131335753674254 1.4754402241238949 0.0 0.0018357448550777998 -0.0017208314230023796 -0.0016281992026439644 0.004053192287174606 0.005653200463096612 0.004838782058285455 0.005642633387268286 0.005642633387268286 0.007254889780984648 0.0 0.002466649165167748 -0.0023547833539290593 -0.002231499474764287 0.00556638790962035 0.007765446739447839 0.006656043961984359 0.007758589418558634 0.007758589418558634 0.009957961029437475 0.5226264436825552 1.744864080275126 1.346071136512628 1.457937727886951 1.9590347181645165 1.9138577235365435 1.7406494272675528 1.8555555839373148 1.8555555839373148 2.224174683913765 0.0 -0.001850490080837831 0.0029442170022884982 0.0027900022070600325 -0.005010850029851647 -0.007212336520829038 -0.006102701205495867 -0.007205964866587455 -0.007205964866587455 -0.009406759340843887 0.527783490053589 1.8664260377445312 1.4349285296408603 1.54641858656526 2.0737686888093174 2.0275711037069484 1.8440385614984842 1.9613813050243898 1.96138130502439 2.3398255807084407 -1.580041573173018 -1.5024792022812363 -1.0276779285351345 -1.2054511638199994 -1.4741159065243852 -1.6050542701560855 -1.7067861803009459 -1.7873387199334256 -1.7873387199334259 -1.962586501937275 -0.6496634928050102 0.8388233574011745 0.8206244378398466 0.7145445532114811 0.5100997042703407 0.2285843462439624 -0.25340042999867235 -0.2812204592057473 -0.2812204592057473 -0.25845740678273227 -0.2273427740329555 -0.16545951440764833 1.0892153387487624 -0.3418457601501167 -0.6411696866991002 -0.764014523180208 -0.7151952736980101 -0.30543569951465643 -0.30543569951465643 -0.5317417265652881 0.21153646458465236 1.4100330148696376 1.8929144353170952 0.9475473014362022 1.0415377838066333 0.8664313076175393 0.7684162329449296 1.2291085877681933 1.2291085877681933 1.3755364247321806 -1.2692102969137347 -1.9863520330427644 -1.5267621183175193 -1.5479813447206532 -1.9890693530052552 -1.987322028290503 -2.0360071491408496 -2.03600714914085 -2.0360071491408496 -2.21810850081579 -0.7720827553963123 0.48186251959320253 0.468260480929101 0.31996019879557386 -0.33553249770605903 -0.3776686538417676 -0.5306654414493829 -0.40751905704348484 -0.40751905704348484 -0.3705811690114644 1.252991322505118 1.4702968864279249 1.0434205651337727 1.1741951217607673 1.6937003881119128 1.689281021859976 1.7242618674370502 1.7243544976147325 1.7243544976147325 1.9307931147421091 1.56797288463705 1.7795795121488507 1.3343872813141677 1.4557966072209416 1.680073765285587 1.8023698113745457 1.8898320250954213 1.9651396724043333 1.9651396724043333 2.129826954481803 0.22439190323874444 0.15348943814844784 -0.7459849198676611 0.5698596444975939 0.847681111742203 0.961266586668062 0.9165172565908981 0.5320376362514646 0.5320376362514646 0.745867387473225 1.08095279614723 -0.7502746874224917`

[rsepassi]

What's the problem you're using? It looks like the vocabulary is not correctly set or is not doing the right thing.

[123mitnik]

using the timeseries toy example problem tensor2tensor/tensor2tensor/data_generators/timeseries.py