ERROR: Index file for var prefix exp-nm-ape/training/variables.data does not exist

[wonkeelee]

Hi, I carefully followed the tutorial of the post-editing task. Although the tutorial has not been updated yet, I have succeeded until the learning step through your help.

However, I faced another new problem again.

to run the test data set on my trained model, I followed the content of tutorial.

The following is from tutorial.

1. Copy the file post-edit.ini into e.g. post-edit.test.ini

2. Open the post-edit.test.ini file and remove the train_dataset and val_dataset sections, as well as the train_dataset and val_dataset configuration from the [main] section.

3. Now we have to make another file specifying the testing dataset configuration. We will call this file post-edit_run.ini

I followed the contents of the tutorial exactly, but it couldn't success to run my trained model with the following error.

2018-01-09 20:40:16: Model built.
2018-01-09 20:40:16: Loading INI file: 'exp-nm-ape/post-edit_run.ini'
2018-01-09 20:40:16: INI file is parsed.
2018-01-09 20:40:16: Building model based on the config.
2018-01-09 20:40:16: Initializing dataset with: translated, source
2018-01-09 20:40:17: Dataset length: 2000
2018-01-09 20:40:17: Model built.

2018-01-09 20:40:17: Default variable file 'exp-nm-ape/training/variables.data' will be used for loading variables.
2018-01-09 20:40:17: Index file for var prefix exp-nm-ape/training/variables.data does not exist
Exception ignored in: <bound method BaseSession.__del__ of <tensorflow.python.client.session.Session object at 0x7f431850d9b0>>

*But I confirmed that the variable files(variables.data.) exist.**

please refer to my three INI files(post-edit.ini(for training), post-edit.test.ini, post-edit_run.ini).

1. post-edit.ini

   
   [main]
   name="post editing"
   output="exp-nm-ape/training"
   runners=[<runner>]
   tf_manager=<tf_manager>
   trainer=<trainer>
   train_dataset=<train_dataset>
   val_dataset=<val_dataset>
   evaluation=[("greedy_edits", "edits", <bleu>), ("greedy_edits", "edits", evaluators.ter.TER)]
   batch_size=128
   runners_batch_size=256
   epochs=160
   validation_period=1000
   logging_period=20
   overwrite_output_dir=True

[tf_manager] class=tf_manager.TensorFlowManager num_threads=4 num_sessions=1 minimize_metric=True save_n_best=3

[bleu] class=evaluators.bleu.BLEUEvaluator name="BLEU-4"

[train_dataset] class=dataset.load_dataset_from_files s_source="exp-nm-ape/data/train/train.src" s_translated="exp-nm-ape/data/train/train.mt" s_edits="exp-nm-ape/data/train/train.edits"

[val_dataset] class=dataset.load_dataset_from_files s_source="exp-nm-ape/data/dev/dev.src" s_translated="exp-nm-ape/data/dev/dev.mt" s_edits="exp-nm-ape/data/dev/dev.edits"

[source_vocabulary] class=vocabulary.from_dataset datasets=[] series_ids=["source"] max_size=8000 save_file="exp-nm-ape/training/vocab/source_vocabulary.tsv" overwrite=True

[target_vocabulary] class=vocabulary.from_dataset datasets=[] series_ids=["edits", "translated"] max_size=10000 save_file="exp-nm-ape/training/vocab/target_vocabulary.tsv" overwrite=True

[src_encoder] class=encoders.recurrent.SentenceEncoder name="src_encoder" rnn_size=300 max_input_len=50 embedding_size=300 dropout_keep_prob=0.8 data_id="source" vocabulary=

[trans_encoder] class=encoders.recurrent.SentenceEncoder name="trans_encoder" rnn_size=300 max_input_len=50 embedding_size=300 dropout_keep_prob=0.8 data_id="translated" vocabulary=

[src_attention] class=attention.Attention name="attention_src_encoder" encoder=

[trans_attention] class=attention.Attention name="attention_trans_encoder" encoder=

[decoder] class=decoders.Decoder conditional_gru=True name="decoder" encoders=[, ] attentions=[, ] rnn_size=300 max_output_len=50 embedding_size=300 dropout_keep_prob=0.5 data_id="edits" vocabulary=

[trainer] class=trainers.cross_entropy_trainer.CrossEntropyTrainer decoders=[] l2_weight=1.0e-8

[runner] class=runners.runner.GreedyRunner decoder= output_series="greedy_edits"

2. post-edit.test.ini

[main] name="post editing" output="exp-nm-ape/training" runners=[] tf_manager= trainer= evaluation=[("greedy_edits", "edits", ), ("greedy_edits", "edits", evaluators.ter.TER)] batch_size=128 runners_batch_size=256 epochs=100 validation_period=1000 logging_period=20 overwrite_output_dir=True

[tf_manager] class=tf_manager.TensorFlowManager num_threads=4 num_sessions=1 minimize_metric=True save_n_best=3

[bleu] class=evaluators.bleu.BLEUEvaluator name="BLEU-4"

[source_vocabulary] class=vocabulary.from_wordlist path="exp-nm-ape/training/vocab/source_vocabulary.tsv"

[target_vocabulary] class=vocabulary.from_wordlist path="exp-nm-ape/training/vocab/target_vocabulary.tsv"

[src_encoder] class=encoders.recurrent.SentenceEncoder name="src_encoder" rnn_size=300 max_input_len=50 embedding_size=300 dropout_keep_prob=0.8 data_id="source" vocabulary=

[trans_encoder] class=encoders.recurrent.SentenceEncoder name="trans_encoder" rnn_size=300 max_input_len=50 embedding_size=300 dropout_keep_prob=0.8 data_id="translated" vocabulary=

[src_attention] class=attention.Attention name="attention_src_encoder" encoder=

[trans_attention] class=attention.Attention name="attention_trans_encoder" encoder=

[decoder] class=decoders.Decoder conditional_gru=True name="decoder" encoders=[, ] attentions=[, ] rnn_size=300 max_output_len=50 embedding_size=300 dropout_keep_prob=0.5 data_id="edits" vocabulary=

[trainer] class=trainers.cross_entropy_trainer.CrossEntropyTrainer decoders=[] l2_weight=1.0e-8

[runner] class=runners.runner.GreedyRunner decoder= output_series="greedy_edits"

3. post-edit_run.ini

[main] test_datasets=[]

[eval_data] class=dataset.load_dataset_from_files s_source="exp-nm-ape/data/test/test.src" s_translated="exp-nm-ape/data/test/test.mt" s_greedy_edits_out="exp-nm-ape/test_output.edits"

[jindrahelcl]

this is weird.. Have you had any progress with this?