valid_world.epoch_done() in train_model.py never finishes

[jojonki]

I checked a memory networks model with babi dataset. But I found valid_world.epoch_done never finishes. https://github.com/facebookresearch/ParlAI/blob/master/examples/train_model.py#L91

• My revision 55ca9c (Jan 11, 2018)

• the log

  $ python examples/train_model.py -t dialog_babi:Task:1 -m memnn -mf /tmp/model_memnn  -e 1
  ...
  [ time:68s parleys:5873 time_left:1s num_epochs:0 ] {'total': 204, 'accuracy': 0.09804, 'f1': 0.1517, 'hits@k': {1: 0.098, 5: 0.333, 10: 0.725, 100: 1.0}}
  [ time:69s parleys:6024 time_left:0s num_epochs:1 ] {'total': 151, 'accuracy': 0.1192, 'f1': 0.1749, 'hits@k': {1: 0.119, 5: 0.377, 10: 0.695, 100: 1.0}}
  [ num_epochs completed:1.0 time elapsed:69.75176429748535s ]
  [ running eval: valid ]
  [creating task(s): dialog_babi:Task:1]
  [loading fbdialog data:/home/jonki/work/ParlAI/data/dialog-bAbI/dialog-bAbI-tasks/dialog-babi-task1-API-calls-dev.txt]
  // never breaks while-loop

[jojonki]

babi task is working. So this is the problem of dialog_babi.

// this is working
python examples/train_model2.py -t babi:task1k:1 -m memnn -mf /tmp/model_memnn  -e 1

In dialog_babi, episode_idx looks always 0. Then epoch_done is always False.

[klshuster]

The memory network operates on the validation set very slowly (it considers many more candidates than it does while training), but it is not hanging. If you give it some time, it should finish. Please reopen if you have any more questions!

[jaseweston]

FYI We are in the process of updating this model to make it faster

On Tue, Jan 16, 2018 at 10:31 AM, Kurt Shuster notifications@github.com wrote:

The memory network operates on the validation set very slowly (it considers many more candidates than it does while training), but it is not hanging. If you give it some time, it should finish. Please reopen if you have any more questions!

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/facebookresearch/ParlAI/issues/498#issuecomment-357998812, or mute the thread https://github.com/notifications/unsubscribe-auth/AKjk-PFQ4MqKnekx_QLHg0wvWQ6wB6SRks5tLMDmgaJpZM4RdGzX .

[jojonki]

@jaseweston Nice. Are there any issues related to that?

[jaseweston]

No.. you want me to create one?

On Mon, Jan 22, 2018 at 2:37 PM, Junki Ohmura notifications@github.com wrote:

@jaseweston https://github.com/jaseweston Nice. Are there any issues related to that?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/facebookresearch/ParlAI/issues/498#issuecomment-359540054, or mute the thread https://github.com/notifications/unsubscribe-auth/AKjk-I1FjRa2FfaploLmWPUXMi_HpdRMks5tNOORgaJpZM4RdGzX .

[jojonki]

It is helpful indeed. And when will you release the faster version? Please let me know if you have a plan in your mind about this.

[alexholdenmiller]

516