Closed vinayakumarr closed 6 years ago
Hi @vinayakumarr ,
IN the program, there are two parameters that affect the checkpoint process:
if step % FLAGS.CHECKPOINT_EVERY == 0 or idx == (nb_batches-1):
if dev_loss < min_dev_loss:
path = saver.save(sess, checkpoint_prefix, global_step = step)
print("Dev loss improved: {} -> {}".format(min_dev_loss, dev_loss))
print("Saved model checkpoint to {}\n".format(path))
min_dev_loss = dev_loss
else:
print("Dev loss did not improve: {} -> {}".format(min_dev_loss, dev_loss))
Note that in the current codes, the model checkpoint is saved only after each evaluation step (EVAL_EVERY) on the validation data and the new validation loss is improved/decreased. You can change this saving process yourself according to your own requirements.
Program works properly, but during training checkpointer is not saved.
IOError: [Errno 2] No such file or directory: 'runs/1000000_emb5_dlm1_32dim_minwf1_1conv3456_1ep/train_1000000_test_500000.txt'