search

d8ahazard / sd_dreambooth_extension

Other
1.87k stars 284 forks source link

Feature Request: Add option to clone Huggingface diffusers instead of converting .ckpt to diffusers for users with low system RAM #152

Closed Miraculix200 closed 2 years ago

Miraculix200 commented 2 years ago

Free Google Colab and SageMaker Studio Lab have low system RAM, which causes to webui to crash when the Dreambooth extension converts a .ckpt to diffusers.

Suggestion: Add a tab to clone a Huggingface model, to enable users with little system RAM to start the training.

Tab content could be:

That doesn't solve the crash after the training, when the extension tries to convert the diffusers to a .ckpt file. But that can't be fixed from the extension. Users would have to manually use a script for conversion, while the webui is not running, like I do here: https://colab.research.google.com/github/Miraculix200/StableDiffusionUI_Colab/blob/main/StableDiffusionUI_Colab.ipynb#scrollTo=j5_IGbmKPQhu

d8ahazard commented 2 years ago

That's not a terrible idea. On first read over this, I thought you wanted to re-incorporate all this stuff to the main UI, which is a bit more of a PITA. Just a stand-alone tab with an isolated function, that I can get behind.

I'm a bit busy today, but I'll see what I can do.

On Sat, Nov 12, 2022 at 9:58 AM Miraculix 200 @.***> wrote:

Free Google Colab and SageMaker Studio Lab have low system RAM, which causes to webui to crash when the Dreambooth extension converts a .ckpt to diffusers.

Suggestion: Add a tab to clone a Huggingface model, to enable users with little system RAM to start the training.

Tab content could be:

  • Huggingface URL (or profile name / model name)
  • Name the trained model should have
  • Huggingface token

That doesn't solve the crash after the training, when the extension tries to convert the diffusers to a .ckpt file. But that can't be fixed from the extension. Users would have to manually use a script for conversion, while the webui is not running.

— Reply to this email directly, view it on GitHub https://github.com/d8ahazard/sd_dreambooth_extension/issues/152, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAMO4NFKJOAQQPE7XLYNTATWH65DFANCNFSM6AAAAAAR6MDPIY . You are receiving this because you are subscribed to this thread.Message ID: @.***>

Miraculix200 commented 2 years ago

Yes, I thought of it as an (optional) alternative to the "Create Model" tab

d8ahazard commented 2 years ago

Implemented with a9bbe70

You'll need to tell me if this still kills RAM for you. Unfortunately, I can't do a direct clone of the directory because then we also have to download the .ckpt files which takes forever, but I'm hoping that saving the pretrained model when loading from URL should be enough.

image

Miraculix200 commented 2 years ago

It works without crashing, if I enter something into the Model Path textbox. The way it is now, it looks like it would automatically use runwayml/stable-diffusion-v1-5 when clicking create. But I had to enter it manually. Otherwise it loaded the .ckpt file which is selected in the "Stable diffusion checkpoint" selector and crash.

Also, for some reason it doesn't start the training, and there is no error message in the console. But I can select the cloned model

Miraculix200 commented 2 years ago

The webui doesn't crash when starting the training btw. I just shows a progress bar for a second which then disappears again. Can I activate debug messages for the plugin somehow?

Edit: Well, it actually didn't clone the model properly. No wonder it doesnt start the training. The "working" folder doesn't have any content. I'll try again

Edit2: It created another "working" folder next to the "test" (model name) folder. I guess that's why

Edit3: That it created the folder in the wrong place isn't reproducible. Maybe it was caused by user error

Edit4: Starting a training still doesn't work, even when I use my method to clone the diffusers from Huggingface, which previously worked. I guess it's because of recent changes. So I assume it would theoretically work, as the cloning was successful.

Maybe it's because it can't install Fairscale 0.4.9 (another version is already installed), but that error previously wasn't a problem.

Miraculix200 commented 2 years ago

Training now works, after starting a new session on Colab. So this issue can be closed, I guess.

Though I suggest making it actually use the ghosted path in the Model Path textbox ("runwayml/stable-diffusion-v1-5"), instead of requiring the user to enter it manually. It looks like it is a default value, but it doesn't act as default value

d8ahazard commented 2 years ago

Training now works, after starting a new session on Colab. So this issue can be closed, I guess.

Though I suggest making it actually use the ghosted path in the Model Path textbox ("runwayml/stable-diffusion-v1-5"), instead of requiring the user to enter it manually. It looks like it is a default value, but it doesn't act as default value

Good suggestion - added. I've also hidden the "offline" settings when cloning from online to prevent confusion.