search

martinkersner / train-CRF-RNN

Train CRF-RNN for Semantic Image Segmentation
Other
199 stars 92 forks source link

Properly resume training. #11

Closed warmspringwinds closed 8 years ago

warmspringwinds commented 8 years ago

Hello.

Could you, please, give an example of how to resume training using your code?

Thank you.

martinkersner commented 8 years ago

Hi @warmspringwinds!

You can try to use this script. However, you should be aware that you have to change _snapshotprefix in solver.prototxt, otherwise it will rewrite your old models.

Cheers,

Martin

warmspringwinds commented 8 years ago

@martinkersner

Thank you so much :+1:

martinkersner commented 8 years ago

In a case you would like to continue training without regular testing your net, you can run this command from your python-scripts directory

../caffe-crfrnn/build/tools/caffe train --solver=$SOLVER --snapshot=$SOLVERSTATE

where $SOLVER is the same solver.prototxt as you used before you paused training and $SOLVERSTATE is the last model state (file with .solverstate extension) that was snapshotted during training.