Error During Testing

[kadir-gunel]

Hello @SwordYork ,

When testing the model python throws a TypeError message : Slice indices must be integers or None .... And then perl throws exception, probably caused by the previous error.

B.R.

[SwordYork]

Hi, Could you please post the detailed error message, I want to know which line caused this error. Does it produce some translations before the error message? You could paste the entire outputs after run python testing.py dcnmt_en2fr_640000, I would check it. Or maybe the test dataset is missing, could you check it

[kadir-gunel]

INFO:main:Model options: {'batch_size': 30, 'beam_size': 12, 'bidir_encoder_depth': 2, 'bleu_script': './data/multi-bleu.perl', 'bos_token': '', 'dec_embed': 64, 'dec_nhids': 1024, 'dump_freq': 20000, 'enc_embed': 64, 'enc_nhids': 1024, 'eos_token': '', 'finish_after': 1000000, 'hook_samples': 2, 'max_src_seq_char_len': 300, 'max_src_seq_word_len': 50, 'max_trg_seq_char_len': 300, 'max_trg_seq_word_len': 50, 'normalized_bleu': True, 'normalized_val': True, 'print_freq': 50, 'reload': False, 'sampling_freq': 50, 'save_freq': 500, 'saveto': 'dcnmt_ar2tr', 'sort_k_batches': 12, 'source_language': 'ar', 'src_data': './data/all.ar-tr.ar.tok.shuf', 'src_dgru_depth': 1, 'src_dgru_nhids': 512, 'src_vocab': './data/vocab.ar-tr.ar.pkl', 'src_vocab_size': 120, 'step_clipping': 1.0, 'stream': 'stream', 'target_language': 'tr', 'test_set': './data/haber-internet.train.tok.bw.ar', 'test_set_grndtruth': './data/haber.tune.tok2.tr.ref0', 'test_set_out': 'haber.tune.tok2.tr.ref0_model_out.txt', 'transition_depth': 1, 'trg_data': './data/all.ar-tr.tr.tok.shuf', 'trg_dgru_depth': 1, 'trg_dgru_nhids': 512, 'trg_igru_depth': 1, 'trg_igru_nhids': 1024, 'trg_vocab': './data/vocab.ar-tr.tr.pkl', 'trg_vocab_size': 120, 'unk_id': 1, 'unk_token': '', 'weight_scale': 0.01} INFO:main:Creating theano variables INFO:main:Building RNN encoder-decoder INFO:main:Building model INFO:main:Initializing extensions INFO:main:Building sampling model INFO:main:Building bleu tester INFO:main:Initializing main loop INFO:sampling:Started Test: start testing: dcnmt_ar2tr_20000

Error Message Starts here:

Traceback (most recent call last): File "testing.py", line 110, in main(configuration, get_test_stream(*configuration), testing_model) File "testing.py", line 94, in main main_loop._run_extensions('before_training') File "/home/kadir/blocks/lib/python3.5/site-packages/blocks/main_loop.py", line 263, in _run_extensions extension.dispatch(CallbackName(method_name), args) File "/home/kadir/blocks/lib/python3.5/site-packages/blocks/extensions/init.py", line 68, in dispatch getattr(self, str(callback_name))(*args) File "/home/kadir/git-repos/DCNMT/sampling.py", line 257, in before_training self._evaluate_model() File "/home/kadir/git-repos/DCNMT/sampling.py", line 293, in _evaluate_model ignore_first_eol=False) File "/home/kadir/blocks/lib/python3.5/site-packages/blocks/search.py", line 318, in search return self.result_to_lists(result) File "/home/kadir/blocks/lib/python3.5/site-packages/blocks/search.py", line 324, in result_to_lists for output, mask in equizip(outputs, masks)] File "/home/kadir/blocks/lib/python3.5/site-packages/blocks/search.py", line 324, in for output, mask in equizip(outputs, masks)] TypeError: slice indices must be integers or None or have an index method Use of uninitialized value $length_reference in numeric eq (==) at ./data/multi-bleu.perl line 148.

I do not think that the problem is test data. Because it happened once and I corrected the file name in config file.

[kadir-gunel]

No translations occur before the error.

[SwordYork]

Thanks! I find that this error is related to Blocks. Python 3.4 will raise a warning as following:

python3.4/site-packages/blocks/search.py:324: VisibleDeprecationWarning: using a non-integer number instead of an integer will result in an error in the future for output, mask in equizip(outputs, masks)]

But Python 3.5 will result an error as your case, I have open an issue on Blocks. I have provided a temporary fix for this error. You could download the new sampling.py. Besides, I think it is better to try the old version of DCNMT for ar-tr.

[kadir-gunel]

Ok, I will also try the older version. Thank you.

[SwordYork]

@kadir-gunel Thanks for the heads-up. The issue has been fixed in Blocks.

[kadir-gunel]

@SwordYork Great! Thanks!