dvschultz / stylegan2

StyleGAN2 fork with some bonus content
http://arxiv.org/abs/1912.04958
Other
63 stars 55 forks source link

Missing pickle file ? #1

Closed barthduthoit closed 4 years ago

barthduthoit commented 4 years ago

I'm having an issue when trying to train StyleGAN on a custom dataset on paperspace. I've done the installation as indicated in the repo through the bash file. I'm not exactly sure which pkl file it can't find ? Here is the stack trace:

Couldn't find valid snapshot, starting over
Local submit - run_dir: results/00006-stylegan2-insect_1024-1gpu-config-f
dnnlib: Running training.training_loop.training_loop() on localhost...
Streaming data using training.dataset.TFRecordDataset...
Dataset shape = [3, 1024, 1024]
Dynamic range = [0, 255]
Label size    = 0
Traceback (most recent call last):
  File "run_training.py", line 198, in <module>
    main()
  File "run_training.py", line 193, in main
    run(**vars(args))
  File "run_training.py", line 126, in run
    dnnlib.submit_run(**kwargs)
  File "/home/paperspace/stylegan2/dnnlib/submission/submit.py", line 343, in submit_run
    return farm.submit(submit_config, host_run_dir)
  File "/home/paperspace/stylegan2/dnnlib/submission/internal/local.py", line 22, in submit
    return run_wrapper(submit_config)
  File "/home/paperspace/stylegan2/dnnlib/submission/submit.py", line 280, in run_wrapper
    run_func_obj(**submit_config.run_func_kwargs)
  File "/home/paperspace/stylegan2/training/training_loop.py", line 149, in training_loop
    resume_pkl, resume_kimg = misc.locate_latest_pkl(dnnlib.submit_config.run_dir_root)
  File "/home/paperspace/stylegan2/training/misc.py", line 55, in locate_latest_pkl
    latest_pickle = allpickles[-1]
IndexError: list index out of range
barthduthoit commented 4 years ago

Figured this one out by changing resume_pkl to a loaded pretrained network in training_loop.py