search

ak9250 / stylegan-art

train stylegan through transfer learning
Other
371 stars 78 forks source link

Saving/Resuming checkpoints #5

Closed mbodge closed 5 years ago

mbodge commented 5 years ago

Hi there, thanks for posting this code...I'm new to ML and StyleGAN has been a big chunk to bite off. Your code really helps

Question: in the Colab file you said the following:

Note: colab will reset after 12 hours make sure to save your model checkpoints to google drive around 6 hours mark (one tick), the second tick will show up before the reset but I have lost work by not being able to save it to drive in time, then go to runtime->reset all runtimes. Now copy your train model back into colab and start training again from the previous checkpoint.

Can you provide guidance on how to do this? I see people doing this for other types of models but I'm not sure how to go about saving this (is it the .pkl file?) and how do I reset the runtime and go back in and load it back to keep training?

Any help would be amazing, thanks!

ak9250 commented 5 years ago

Hi, two ways, one is you go the the file tab in colab and double click right click on the pkl file to download it or save it to your google drive and mount your drive to again to reload the checkpoint. you set the model to load in training_loop.py which is shown in the notebook

mbodge commented 5 years ago

Thanks for the quick response...do i need to go through all of the steps in the notebook again (ie: searching for photos for the dataset etc)?

On Sat, Jun 8, 2019 at 8:50 PM ak9250 notifications@github.com wrote:

Hi, two ways, one is you go the the file tab in colab and double click right click on the pkl file to download it and you set the model to load in training_loop.py which is shown in the notebook

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/ak9250/stylegan-art/issues/5?email_source=notifications&email_token=AAOTT4WQLFQWNBPRINCQYMLPZR42BA5CNFSM4HWIRXVKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODXIC6HA#issuecomment-500182812, or mute the thread https://github.com/notifications/unsubscribe-auth/AAOTT4WS362PQWWROCMZN6DPZR42BANCNFSM4HWIRXVA .