SDXL Checkpoint Saving has massive structural issues (workaround included)

So on the surface when you save SDXL checkpoints, you can continue to merge them in comfyui, and render images without problems. However if you load them up in kohya to fine-tune, or use them in Automatic1111's checkpoint merger tab, or use the stable-diffusion-model-toolkit you get tons of errors.

So here's the rabbit hole I went down and the dirty fix that seems to allow the model to work in everything. It would be great though if it would save structurally sound SDXL models somehow in order to remove the need for the workaround.

Test Case Workflow:

Kohya throws the following error:

The SD Toolkit Automatic1111 extension throws this. It thinks that it is a SD 1.5 VAE and want to prune most of the data:

In the advanced tab, it shows that it is actually missing SDXL classes as well as positional IDs: advanced_tab.txt

When testing non-comfyui saved checkpoints it would show something like this:

Automatic1111 Checkpoint merger tab initially throws this error: https://github.com/AUTOMATIC1111/stable-diffusion-webui/issues/10460

I then tried this comment:

Which then resulted in this error:

So that lead me to believe that it doesn't like the structure, as well as the metadata.

So I then trimmed the metadata with this script from a friend: https://gist.github.com/rvkwi/2df007fa3d6249474e18a7aa47d42507

Now I can use the checkpoint merger tab just fine. However the structure remains broken if I just resave it.

So the 2nd step of the workaround besides trimming the metadata was to merge with the base SDXL model again in position A with weighted_sum at 0.999999999999999999999. Example:

Now the toolkit correctly shows me this:

Any help getting this to work just in comfyui would be awesome, and I hope this little rabbit hole helps deduce the problem.

szhublox / ambw_comfyui

SDXL Checkpoint Saving has massive structural issues (workaround included) #4