Open WhenMelancholy opened 1 year ago
@WhenMelancholy This happened for me aswell, as far as I know this indicates that the number of images in your training data is not evenly divisible by the number of CUDA devices you're training on. This should only have a negligible impact on training as long as you're only training on one server. I believe this is a warning from PyTorch lightning.
@benearnthof This happened for me aswell, could you please tell me how to debug? Is it because the dataset is not divisible by 16?
There is no reason to debug anything as this warning just indicates some minor inefficiencies when scaling images. My prior statement may be incorrect as this most likely stems from one of the image dimensions not being divisible by 16. This should not impact the model however
During the training process, I encountered the following warning outputs:
Although it did not affect the training, I am unclear about the reason behind this. My training instructions are as follows:
These instructions are referenced from
train_vqgan.sh
.Thank you in advance!