search

drawthingsai / draw-things-community

The community repository for the Draw Things app.
https://drawthings.ai
GNU General Public License v3.0
94 stars 9 forks source link

SDXL Loras imported in versions after 1.20240515.0 generate inaccurate images #8

Closed Morac2 closed 4 months ago

Morac2 commented 4 months ago

I reported this on Discord, but wanted to report it here as well.

I accidentally imported a tensorsafe Lora today I had imported back on May 17. It didn't overwrite even though the names were the same so I went to look at the files to see which was which and found that the newly imported one was 20% smaller than the old one: 178 MB vs 224 MB. The original file was 218 MB, so the new import was significantly smaller than the original file instead of larger.

I checked my Loras and all the SDXL ones I imported since June 2 were smaller than expected. Using Time Machine I went back to an older version (1.20240515.0) and imported the Lora and found it imported at the larger size. I then started going forward to all saved versions and 1.20240603.0 was the first one I found that generated the smaller import file. I didn't have 1.20240531.1 saved though so this first started happening in either 1.20240531.1 or 1.20240603.0. I see version 1.20240531.1 mentioned it fixed an issue importing SDXL ControlNETs. Maybe it impacted Lora imports as well?

This only impacts SDXL Loras. SD1.5 imports had the same size in all versions. Also the files created when importing SDXL models were identical in old and new versions of Draw Things.

In addition to the smaller checkpoint file size, most of the Loras were not generating images that they were designed to do. They would generate images, but many wouldn't generate images that looked like what the Lora indicated it should. Some were close, others were way off. Importing these Loras using Draw Things 1.20240515.0, I could then generate much better quality images in version 1.20240622.0, which is the latest as of writing this.

Here's several examples of images generated with the same configuration and seeds. The only difference is which version of Draw Things they were imported under. All images were generated in version 1.20240622.0 on a MacBook Air M1 (2020) with 8 GB RAM running MacOS 14.5

https://civitai.com/models/459734/dolores-madrigal-encanto-ponyxl New: score_9__score_8_up__score_7_up__score_6_up__score_5_up__score_4_up__1girl__dolores__head_tilted__mountain_background_162451512

Old (this is what the image should look like): score_9__score_8_up__score_7_up__score_6_up__score_5_up__score_4_up__1girl__dolores__head_tilted__mountain_background_162451512(1)

https://civitai.com/models/376778?modelVersionId=420749 New (NVIDIA seed): image2

Old (NVIDIA seed, this is what the image should look like): pov___lora_riley_andersen_000015_0_8__r_ly__1girl__skinny__blonde__blue_eyes___looking_at_viewer__squatting__front_view__spreading_legs___score_9__score_8_up__score_7_up__-score_6__scor_2749102160

https://civitai.com/models/529325?modelVersionId=588201 New (not that bad though it's oversaturated): brittney__1girl__ponytail__solo__brown_hair__pink_dress__blue_eyes__smile__yellow_scarf__yellow_leg_warmers__sleeveless__purple_pantyhose__blush___score_9__score_8_up__score_7_up__sc_2154228062(1)

Old: brittney__1girl__ponytail__solo__brown_hair__pink_dress__blue_eyes__yellow_scarf__yellow_leg_warmers__sleeveless_shoes__purple_pantyhose__lora_brittneypony1_0_1__brittney__1girl__pon_2154228062(1)

Morac2 commented 4 months ago

It seems like slider Loras aren't impacted by this for some reason.

liuliu commented 4 months ago

Should be fixed in https://github.com/drawthingsai/draw-things-community/commit/f5d3cee7ecd5fe72ad5cd48053933e8f412a4ce6

The LoRA on text encoder 2 is not properly imported for these LoRAs (due to they use transformers formulation, and the change previously is to make it compatible with openclip repo's formulation).