ai-forever / ru-dalle

Generate images from texts. In Russian
https://rudalle.ru/
Apache License 2.0
1.64k stars 246 forks source link

Fine-tuning notebook no longer saves model #115

Closed StephenHnilica closed 2 years ago

StephenHnilica commented 2 years ago

Fine-tuning notebook no longer saves model. Not sure why. It's happening on my private notebooks & the pubic one too.

I did verify that the 2.63 gb model downloads, but /checkpoints/ folder is never created.

`Malevich is 1.3 billion params model from the family GPT3-like, that uses Russian language and text+image multi-modality.

FileNotFoundError Traceback (most recent call last) in () 2 model_path = os.path.join('/content/'+args.save_path,f"{args.model_name}_dalle_last.pt") 3 model = get_rudalle_model('Malevich', pretrained=True, fp16=True, device=device) ----> 4 model.load_state_dict(torch.load(model_path)) 5 print(f'Loaded from {model_path}') 6 vae = get_vae().to(device)

2 frames /usr/local/lib/python3.7/dist-packages/torch/serialization.py in init(self, name, mode) 209 class _open_file(_opener): 210 def init(self, name, mode): --> 211 super(_open_file, self).init(open(name, mode)) 212 213 def exit(self, *args):

FileNotFoundError: [Errno 2] No such file or directory: '/content/checkpoints/awesomemodel__dalle_last.pt'`

StephenHnilica commented 2 years ago

Never mind, I used cd to change directories and never changed it back.