search

aszc-dev / ComfyUI-CoreMLSuite

A set of custom nodes for ComfyUI that allow you to use Core ML models in your ComfyUI workflows.
GNU General Public License v3.0
131 stars 12 forks source link

Loras causes model to produce garbage #36

Open skuznyuk opened 10 months ago

skuznyuk commented 10 months ago

When I have no LoRa everything works perfectly fine. However when I added LoRa to the conversion the output is always garbage (shown in picture). Is there something I did wrong in my workflow with regard to LoRa?

Is LoRa not supported in turbo SDXL models conversions? This LoRa (Trained on non-turbo though) works perfectly fine with this model in DrawThings so I dont think its a LoRa/Model compatibility issue.

Screenshot 2024-01-30 at 12 09 32 PM
aszc-dev commented 10 months ago

Unfortunately, I am unable to replicate your issue with the models/LoRAs I'm using. I've downloaded the same model and used with some other LoRA and the converted model produced nearly exact copy of safetensors model's image. I'm not sure what the problem may be. The only solution that comes to my mind for now is to try different LoRAs and see if that works.

skuznyuk commented 10 months ago

Thank you for the quick response. I attempted with another LoRA and got same result. https://civitai.com/models/122359/detail-tweaker-xl

image

Are you able to provide a screenshot of your workflow, maybe I am missing something.

kartikagrawal2503 commented 7 months ago

I also faced the same when I tried to apply the same way given in Readme. But later on when I tried something experimental It worked. But still I am not 100% sure whether Lora models are working or it is just checkpoint.

aszc-dev commented 7 months ago

I've tried some LoRAs and couldn't get them to working, even those that used to work. I will need to figure out what is causing this.

@skuznyuk @kartikagrawal2503 If you find a LoRA that is working and don't mind sharing, please update me with the details. It might help me resolving the issue.

kartikagrawal2503 commented 7 months ago

Hi @aszc-dev , I am able to use below LoRA: https://civitai.com/models/58390/detail-tweaker-lora-lora

I am not using it the way described in workflow as it was not working. Instead applying via CoreML adapter.