training on .pth files

[x33rak]

@Ephemeral182 Is there a way to use the pretrained .pth files such as (udrs2former_agan.pth) in the training loop for further training. From what I have run, the training loop requires a checkpoint file with epoch, lr_optimizer, etc info which the .pth does not contain.

Is there any way to get the checkpoints file or hyperparameters of the final udrs2former_agan.pth file that your included in you github repo. Thanks.

[Ephemeral182]

Thanks for your attention. For your question, you can use self.model.load_state_dict(torch.load('your .pth checkpoint')) in init part, instead of using 'resume==True'

[x33rak]

@Ephemeral182 which file contains the resume==True flag, I have checkws all of the files and do not see this anywhere. Would it be the coolsystems init function in the train.py file?

[Ephemeral182]

Sorry, I misremembered. I am now using the new version of the pytorch_lightning template. Since I manually took out the model weights when open source to facilitate inference, it is difficult to resume the entire process. The alternative method is to initialize load the model's weights in coolsystems init, but this may result in the need for more eps because the hyperparameters are re-parameterized.

[x33rak]

I warm started the model using the checkpoint file from 1 epoch and replaced the weights of the 1 epoch model with the weights of the .pth file. I am unsure if this is the correct way to do it but it seems to be training (although the training is unstable in the beginning).