Open odettecantswim opened 1 year ago
same problem, anyone help?
Are you sure you properly mounted your drive?
the program usually makes a checkpoint at 1000 and should output this entry when it does.
2023-09-01 05:42:42,607 checkpoints INFO Saving model and optimizer state at iteration 334 to ../drive/MyDrive/vits-finetune/checkpoints/G_1000.pth
2023-09-01 05:42:45,077 checkpoints INFO Saving model and optimizer state at iteration 334 to ../drive/MyDrive/vits-finetune/checkpoints/D_1000.pth
this is what the program logs when you are close, do note the "800" as it has a default log interval of every 200 reached.
2023-09-01 05:11:31,800 checkpoints INFO Train Epoch: 267 [67%]
2023-09-01 05:11:31,802 checkpoints INFO [1.0736439228057861, 4.700018882751465, 25.126644134521484, 18.40508270263672, 1.3811384439468384, 2.039275884628296, 800, 0.00017838340910211005]
i trained a model and the runtime said it saved the checkpoint at 33, 66 and 100 epoch, but the checkpoint doesn't appears on my drive path, it only had the logs of it. i executed all of the code and put the training dataset correctly. is there any solution for this problem?