demo.py doesn't find experiment

[julianlanger]

Dear all,

I am just trying out your algorithm. To speed up with my new data, I set both n_epochs and evaluate_every_epoch to 1. The training part also runs through. However, if I then run demo.py to look at the processed images, it throws the following error.

model_dir = os.path.join(model_base_dir, max(possible_dirs)) # Take latest export ValueError: max() arg is an empty sequence

It seems that demo.py doesn't find the trained model, although I find the following files in page_model folder:

checkpoint config.json eval events.out.tfevents.1565610667.newspaper-vm export graph.pbtxt model.ckpt-800.data-00000-of-00001 model.ckpt-800.index model.ckpt-800.meta

What should I do?

[solivr]

Hello @julianlanger,

It's seems that tf.estimator.BestExporter doesn't always export a model after the first epoch, however it's not clear why... We can add a condition at the end of the training (train.py) that checks if the export folder is empty and uses a tf.estimator.FinalExporter to export the last model.

# If export directory is empty, export a model anyway
if not os.listdir(saved_model_dir):
    final_exporter = tf.estimator.FinalExporter(name='FinalExporter',  serving_input_receiver_fn=serving_input_fn)
    final_exporter.export(estimator, saved_model_dir, checkpoint_path=None, eval_result=eval_result, is_the_final_export=True)

Or you can train your model a bit longer. ;)