Switching the Encoder Gives Error

[ghost]

Hello,

In the seq2seq_attn example, I tried replacing the encoder with hierarchical encoder from the hred example.

I used the same hparams as well, but I still get error.

encoder = tx.modules.HierarchicalRNNEncoder(hparams=encoder_hparams)

How to replace the encoder, decoder ? Do the code need to be re-written if we want to change just one part?

thank you.

[ZhitingHu]

can you post the error msg?

[ghost]

@ZhitingHu Thank you for the response. Here is the error I am getting for the problem:

python seq2seq_attn.py --config_model config_model_full --config_data config_iwslt14
Traceback (most recent call last):
  File "seq2seq_attn.py", line 210, in <module>
    main()
  File "seq2seq_attn.py", line 109, in main
    train_op, infer_outputs = build_model(batch, train_data)
  File "seq2seq_attn.py", line 55, in build_model
    enc_outputs, _ = encoder(source_embedder(batch['source_text_ids']))
  File "/home/text_sum/texar/texar/module_base.py", line 116, in __call__
    return self._template(*args, **kwargs)
  File "/home/anaconda2/envs/tfmr/lib/python3.6/site-packages/tensorflow/python/ops/template.py", line 351, in __call__
    result = self._call_func(args, kwargs)
  File "/home/anaconda2/envs/tfmr/lib/python3.6/site-packages/tensorflow/python/ops/template.py", line 302, in _call_func
    result = self._func(*args, **kwargs)
  File "/home/text_sum/texar/texar/modules/encoders/hierarchical_encoders.py", line 253, in _build
    inputs = tf.reshape(inputs, shape + [inputs.shape[3]])
  File "/home/anaconda2/envs/tfmr/lib/python3.6/site-packages/tensorflow/python/framework/tensor_shape.py", line 524, in __getitem__
    return self._dims[key]
IndexError: list index out of range

originally defined at:
  File "seq2seq_attn.py", line 53, in build_model
    hparams=config_model.hierarchical_encoder)
  File "/home/text_sum/texar/texar/modules/encoders/hierarchical_encoders.py", line 68, in __init__
    EncoderBase.__init__(self, hparams)
  File "/home/text_sum/texar/texar/modules/encoders/encoder_base.py", line 33, in __init__
    ModuleBase.__init__(self, hparams)
  File "/home/text_sum/texar/texar/module_base.py", line 73, in __init__
    create_scope_now_=True)
  File "/home/anaconda2/envs/tfmr/lib/python3.6/site-packages/tensorflow/python/ops/template.py", line 152, in make_template
    **kwargs)

[ZhitingHu]

can you check what shape batch['source_text_ids'] has? It should be of 3 dimensions.

Have you set "variable_utterance": True in your data config (see example)?

[ghost]

@ZhitingHu thanks for the reply. no, i dont't have "variable_utterance": True in data config.

But Isn't hierarchical rnn an "encoder" by itself, i.e. the rnn up on the hierarchy will operate with larger length time steps than the rnn below? does variable utterance parameter controls that property?

How to set "variable_utterance": True, e.g. for the iwslt example in the seq2seq_attn code?

[ZhitingHu]

HierarchicalRNNEncoder requires a 4D inputs (see docs). The encoder can be used for, e.g., encoding dialog history.

I don't think it's generally necessary to use a hierarchical encoder for sentence translation