search

daveschumaker / artbot-for-stable-diffusion

A front-end GUI for interacting with the AI Horde / Stable Diffusion distributed cluster
https://tinybots.net/artbot
MIT License
172 stars 40 forks source link

Incompatibility Issue: LoRa Modules labeled as 'other' not Functioning in ARTBOT #143

Closed wesleywil closed 1 year ago

wesleywil commented 1 year ago

I've come across this problem in the ARTBOT project, it's related to the utilization of the LORA modules.

Issue details:

The lora models in CIVITAI has in the right side of the screen in the details category an attribute called "base model", every time this attribute is set to "other" in the CIVITAI the functionality of the LORA module does not work in the ARTBOT, but when the "base model" is set to SD 1.5 for example, the lora works fine

Steps to reproduce(PROBLEM):

  1. Model with base model set to "other" -> https://civitai.com/models/122921/anne-hathaway
  2. Integrate the Lora module into the ARTBOT system
  3. Generate the AI art

Lora working from the same author:

  1. Model with base model set to "SD 1.5" ->https://civitai.com/models/124919/elizabeth-bioshock-cosplay
  2. Integrate the Lora module into the ARTBOT system
  3. Generate the AI art

Additional Notes:

I chose this lora creator(Digital_Art_AI) in the CIVITAI, because he creates the loras the same way always, the only difference was in the process of configuration of his lora in the site where he changed the attribute "base model" from "SD 1.5" to "other". This minor change is what makes this kind of lora to not work in ARTBOT

I suggest that if a LoRa module in CIVITAI has the 'base model' set as 'other,' we should use SD 1.5 in any given configuration used in the ARTBOT. Since most of the LoRa models are trained on SD 1.5, this would be only a workaround. However, those loras that were trained with SD 2.0 or SD 2.1 might still have the same issue.

db0 commented 1 year ago

This is not an artbot problem. The workers will refuse to use a Lora whose baseline does not match the main checkpoint because it can crash the worker. There's nothing we can do. The author has to simply mark their Lora correctly

wesleywil commented 1 year ago

This is not an artbot problem. The workers will refuse to use a Lora whose baseline does not match the main checkpoint because it can crash the worker. There's nothing we can do. The author has to simply mark their Lora correctly

That's make sense, is like a security feature for the workers, ty