search

OSU-NLP-Group / MagicBrush

[NeurIPS'23] "MagicBrush: A Manually Annotated Dataset for Instruction-Guided Image Editing".
https://osu-nlp-group.github.io/MagicBrush/
Other
308 stars 14 forks source link

Issue when loading the model #4

Closed giuseppecartella closed 1 year ago

giuseppecartella commented 1 year ago

Hi, I am trying to run your finetuned instructpix2pix model. Following your instructions I downloaded the checkpoint and then used the source code from instructpix2pix repo.

When running edit_cli.py, the program remains blocked indefinitely when executing the instruction torch.load(ckpt). I am running on a 12GB gpu and using 24GB of RAM. No cuda out of memory error emerged at the moment.

Do you have any tips to solve the issue? Thank you! Giuseppe

drogozhang commented 1 year ago

Hi Giuseppe,

Yes, it would take a very long time for loading. For me, it took ~1 min (one A6000 with 48GB). And the same "issue" happened for the original InstructPix2Pix checkpoint.

I would recommend to use the edit_app so the model will be loaded and work for a long time and multiple images. Using cloud GPU and deploying models there is a commonly used technique for diffusion models (edit_app will get you a public link so you can edit image locally with the cloud GPU).

Help it helps :)

Best, Kai

giuseppecartella commented 1 year ago

Thank you so much for your valuable feedback.