should we use t2t-decoder in classification problem

[xusong123]

I wonder if we should use t2t-decode in classification problem like sentiment_imdb:

When I try to decode to get the classification label, it raises an exception: AttributeError: 'HParams' object has no attribute 'problem_hparams'

my script is like this:

PROBLEM=sentiment_imdb
MODEL=transformer_encoder
HPARAMS=transformer_tiny

DATA_DIR=$HOME/t2t_data
TMP_DIR=/tmp/t2t_datagen
#TRAIN_DIR=./t2t_train/$PROBLEM/$MODEL-$HPARAMS
TRAIN_DIR=$HOME/t2t_train/$PROBLEM/$MODEL-$HPARAMS

DEVICES=1

CUDA_VISIBLE_DEVICES=$DEVICES t2t-trainer \
  --data_dir=$DATA_DIR \
  --problem=$PROBLEM \
  --model=$MODEL \
  --hparams_set=$HPARAMS \
  --output_dir=$TRAIN_DIR \
  --train_steps=2000 \
  --generate_data

DECODE_FILE=$DATA_DIR/classify_this.txt
echo "Hello world" >> $DECODE_FILE
echo "Goodbye world" >> $DECODE_FILE
#echo -e 'Hallo Welt\nAuf Wiedersehen Welt' > ref-translation.de

BEAM_SIZE=1
#ALPHA=0.6

CUDA_VISIBLE_DEVICES=$DEVICES t2t-decoder \
  --data_dir=$DATA_DIR \
  --problems=$PROBLEM \
  --model=$MODEL \
  --hparams_set=$HPARAMS \
  --output_dir=$TRAIN_DIR \
  --decode_hparams="beam_size=$BEAM_SIZE" \
  --decode_from_file=$DECODE_FILE \
  --decode_to_file=classify_result

aftern training done, it raises an AttributeError in t2t-decoder:

................................
INFO:tensorflow:Finished evaluation at 2018-07-13-06:53:54
INFO:tensorflow:Saving dict for global step 6002: global_step = 6002, loss = 0.2245153, metrics-sentiment_imdb/targets/accuracy = 0.8193431, metrics-sentiment_imdb/targets/accuracy_per_sequence = 0.8193431, metrics-sentiment_imdb/targets/accuracy_top5 = 1.0, metrics-sentiment_imdb/targets/approx_bleu_score = 0.9492794, metrics-sentiment_imdb/targets/neg_log_perplexity = -0.4349131, metrics-sentiment_imdb/targets/rouge_2_fscore = 0.0, metrics-sentiment_imdb/targets/rouge_L_fscore = 0.8170159
INFO:tensorflow:Stop training model as max steps reached
INFO:tensorflow:schedule=continuous_train_and_eval
INFO:tensorflow:worker_gpu=1
INFO:tensorflow:sync=False
WARNING:tensorflow:Schedule=continuous_train_and_eval. Assuming that training is running on a single machine.
INFO:tensorflow:datashard_devices: ['gpu:0']
INFO:tensorflow:caching_devices: None
INFO:tensorflow:ps_devices: ['gpu:0']
INFO:tensorflow:Using config: {'data_parallelism': <tensor2tensor.utils.expert_utils.Parallelism object at 0x7f7ac6fad588>, '_master': '', '_is_chief': True, '_save_summary_steps': 100, '_save_checkpoints_secs': None, '_log_step_count_steps': 100, '_tf_config': gpu_options {
  per_process_gpu_memory_fraction: 1.0
}
, '_keep_checkpoint_every_n_hours': 10000, '_task_id': 0, '_keep_checkpoint_max': 20, '_tf_random_seed': None, '_model_dir': '/home/xuhengru/t2t_train/sentiment_imdb/transformer_encoder-transformer_tiny', 't2t_device_info': {'num_async_replicas': 1}, '_environment': 'local', '_evaluation_master': '', '_save_checkpoints_steps': 1000, '_num_ps_replicas': 0, '_cluster_spec': <tensorflow.python.training.server_lib.ClusterSpec object at 0x7f7a5652beb8>, '_task_type': None, 'use_tpu': False, '_num_worker_replicas': 0, '_session_config': gpu_options {
  per_process_gpu_memory_fraction: 0.95
}
allow_soft_placement: true
graph_options {
  optimizer_options {
  }
}
}
WARNING:tensorflow:Estimator's model_fn (<function T2TModel.make_estimator_model_fn.<locals>.wrapping_model_fn at 0x7f7a4cf999d8>) includes params argument, but params are not passed to Estimator.
INFO:tensorflow:decode_hp.batch_size not specified; default=32
Traceback (most recent call last):
  File "/home/xuhengru/pytf/bin/t2t-decoder", line 16, in <module>
    tf.app.run()
  File "/home/xuhengru/pytf/lib/python3.5/site-packages/tensorflow/python/platform/app.py", line 126, in run
    _sys.exit(main(argv))
  File "/home/xuhengru/pytf/bin/t2t-decoder", line 12, in main
    t2t_decoder.main(argv)
  File "/home/xuhengru/pytf/lib/python3.5/site-packages/tensor2tensor/bin/t2t_decoder.py", line 190, in main
    decode(estimator, hp, decode_hp)
  File "/home/xuhengru/pytf/lib/python3.5/site-packages/tensor2tensor/bin/t2t_decoder.py", line 92, in decode
    checkpoint_path=FLAGS.checkpoint_path)
  File "/home/xuhengru/pytf/lib/python3.5/site-packages/tensor2tensor/utils/decoding.py", line 257, in decode_from_file
    p_hp = hparams.problem_hparams
AttributeError: 'HParams' object has no attribute 'problem_hparams'

[ptakopysk]

I also cannot get text classification work on my own inputs. I tried the poem line generation from the tutorial, this works fine. I then modified it to be Text2Class instead of Text2Text, predicting the length of the line as the "class". The training works fine, but when I try to use it on a text file, it dies. Only in my case, the errors are something like:

INFO:tensorflow:Cannot use 'while/Pad' as input to 'Identity_49' because 'while/Pad' is in a while loop.

Identity_49 while context: None while/Pad while context: while/while_context

Traceback for Identity_49: File "/lnet/spec/tmp/rosa/t2t/venv/bin/t2t-decoder", line 17, in tf.app.run() File "/lnet/spec/tmp/rosa/t2t/venv/lib/python3.6/site-packages/tensorflow/python/platform/app.py", line 40, in run _run(main=main, argv=argv, flags_parser=_parse_flags_tolerate_undef) File "/lnet/spec/tmp/rosa/t2t/venv/lib/python3.6/site-packages/absl/app.py", line 299, in run _run_main(main, args) File "/lnet/spec/tmp/rosa/t2t/venv/lib/python3.6/site-packages/absl/app.py", line 250, in _run_main sys.exit(main(argv)) File "/lnet/spec/tmp/rosa/t2t/venv/bin/t2t-decoder", line 12, in main t2t_decoder.main(argv) File "/lnet/spec/tmp/rosa/t2t/venv/lib/python3.6/site-packages/tensor2tensor/bin/t2t_decoder.py", line 205, in main decode(estimator, hp, decode_hp) File "/lnet/spec/tmp/rosa/t2t/venv/lib/python3.6/site-packages/tensor2tensor/bin/t2t_decoder.py", line 94, in decode checkpoint_path=FLAGS.checkpoint_path) File "/lnet/spec/tmp/rosa/t2t/venv/lib/python3.6/site-packages/tensor2tensor/utils/decoding.py", line 474, in decode_from_file for elapsed_time, result in timer(result_iter): File "/lnet/spec/tmp/rosa/t2t/venv/lib/python3.6/site-packages/tensor2tensor/utils/decoding.py", line 468, in timer item = next(gen) File "/lnet/spec/tmp/rosa/t2t/venv/lib/python3.6/site-packages/tensorflow_estimator/python/estimator/estimator.py", line 619, in predict features, None, ModeKeys.PREDICT, self.config) File "/lnet/spec/tmp/rosa/t2t/venv/lib/python3.6/site-packages/tensorflow_estimator/python/estimator/estimator.py", line 1146, in _call_model_fn model_fn_results = self._model_fn(features=features, kwargs) File "/lnet/spec/tmp/rosa/t2t/venv/lib/python3.6/site-packages/tensor2tensor/utils/t2t_model.py", line 1415, in wrapping_model_fn use_tpu=use_tpu) File "/lnet/spec/tmp/rosa/t2t/venv/lib/python3.6/site-packages/tensor2tensor/utils/t2t_model.py", line 1472, in estimator_model_fn return model.estimator_spec_predict(features, use_tpu=use_tpu) File "/lnet/spec/tmp/rosa/t2t/venv/lib/python3.6/site-packages/tensor2tensor/utils/t2t_model.py", line 1708, in estimator_spec_predict features[name] = tf.identity(feature) File "/lnet/spec/tmp/rosa/t2t/venv/lib/python3.6/site-packages/tensorflow/python/util/dispatch.py", line 180, in wrapper return target(*args, *kwargs) File "/lnet/spec/tmp/rosa/t2t/venv/lib/python3.6/site-packages/tensorflow/python/ops/array_ops.py", line 86, in identity ret = gen_array_ops.identity(input, name=name) File "/lnet/spec/tmp/rosa/t2t/venv/lib/python3.6/site-packages/tensorflow/python/ops/gen_array_ops.py", line 4253, in identity "Identity", input=input, name=name) File "/lnet/spec/tmp/rosa/t2t/venv/lib/python3.6/site-packages/tensorflow/python/framework/op_def_library.py", line 788, in _apply_op_helper op_def=op_def) File "/lnet/spec/tmp/rosa/t2t/venv/lib/python3.6/site-packages/tensorflow/python/util/deprecation.py", line 507, in new_func return func(args, kwargs) File "/lnet/spec/tmp/rosa/t2t/venv/lib/python3.6/site-packages/tensorflow/python/framework/ops.py", line 3616, in create_op op_def=op_def) File "/lnet/spec/tmp/rosa/t2t/venv/lib/python3.6/site-packages/tensorflow/python/framework/ops.py", line 2005, in init self._traceback = tf_stack.extract_stack()