Flux metadata lists the previously loaded checkpoint

Taruvi commented 1 month ago

Expected Behavior

When looking at the metadata for previous image generations, the actual checkpoint used for the generation should be listed.

Actual Behavior

The metadata for images generated with Flux still list the previously loaded checkpoint.

Steps to Reproduce

After loading the provided example workflow for Schenll and sending it to the Generate tab:

User comment    {
  "sui_image_params": {
    "model": "sdxlpony/DucHaiten-Pony-Real - Pony-Real_v10",
    "aspectratio": "Custom",
    "width": 1024,
    "height": 1024,
    "comfyrawworkflowinputtextcliptextencodenodetextg": "a bottle with a rainbow galaxy inside it on top of a wooden table on a table in the middle of a modern kitchen",
    "automaticvae": true,
    "seed": 1332356456,
    "prompt": "",
    "negativeprompt": "",
    "swarm_version": "0.9.1.1",
    "date": "2024-08-02",
    "generation_time": "0.01 (prep) and 17.08 (gen) seconds"
  }
}

When checking all the options on the left, their metadata are listed but it still lists the previous checkpoint.

Examples:

Screenshot 2024-08-02 at 13-33-42 Image Generation - SwarmUI

Screenshot 2024-08-02 at 13-33-57 Image Generation - SwarmUI

Debug Logs

Not relevant.

Other

No response

mcmonkey4eva commented 1 month ago

This isn't a bug, this is a jankily configured custom comfy workflow. You do in fact have a stray pony model loading there, as the comfy workflow isn't taking on the Swarm model parameter (normally this automates itself with CheckpointLoaderSimple node, but Flux uses the UNETLoader node so it doesn't automate atm)

Taruvi commented 1 month ago

I see. So whatever model I last loaded will always be (apparently) there while I'm using Flux? I don't see any way to kind of unload it manually.

Either way - not really a problem for now. In the future it could be if I'm trying to recreate something or compare Flux.1 with Flux.2. Something like that. Also, it makes filtering in Image History awkward. I'm making sure to set the settings sliders to 'on' just so I can have the word 'flux' in there to filter with, even if it just says 'textunetloadernodeunetnamem: flux1-schnell.sft' as that at least has the model name and version in it. Except if I go to filter for the other model, it also returns these Flux gens as well.

mcmonkey4eva commented 1 month ago

I see. So whatever model I last loaded will always be (apparently) there while I'm using Flux? That is not what I said at all?

This is a problem specific to the comfy workflow setup you have, and is fixed by either just not doing the custom workflow, or by linking the swarm model param (Just make a primitive SwarmUI: Model and link it to the unetloader input)

Taruvi commented 1 month ago

That is not what I said at all?

Maybe I didn't word it so well. I meant it looks like it's apparently there when I filter images. As in - it's there in the metadata. I didn't mean that it's still being loaded. As you said - jank.

I'm using this workflow because I have it nicely hooked up with ReActor and it's all working perfectly well like that.

I tried doing what you say - adding a primitive and (after changing the unet_name to an input) hooking it up like that but I get an error:

Refused to generate image for local: ComfyUI execution error: 'NoneType' object has no attribute 'lower'

It looks like this:

SwarmUI

Is this right? Or wrong? It's probably wrong, isn't it? I mean it seems like what you said... But I'm still a total noob at Comfy and all its node naming schemes. That's why I'm using as little of Comfy for now and as much of Swarm as I can. At least until I get a handle on Comfy things.

mcmonkey4eva commented 1 month ago

That looks correct. I can't diagnose that error without more info.

I just did myself to test:

and it works as intended:

This is on a very clean workflow though, so there might be something more to what's going wrong. It might be in the swarm interface you have the wrong model selected? It should autoswap to the one in your workflow but sometimes that doesn't happen right

Taruvi commented 1 month ago

I've deselected everything in the interface in case that was trying to override something.

This is Debug log when I click Generate:

2024-08-03 00:48:07.249 [Info] User local requested 8 images with model 'sdxlpony/DucHaiten-Pony-Real - Pony-Real_v10.safetensors'...
2024-08-03 00:48:07.250 [Debug] [BackendHandler] Backend request #282 for model sdxlpony/DucHaiten-Pony-Real - Pony-Real_v10.safetensors, maxWait=7.00:00:00.
2024-08-03 00:48:07.250 [Debug] [BackendHandler] Backend request #282 found correct model on #0
2024-08-03 00:48:07.250 [Debug] [BackendHandler] Backend request #282 finished.
2024-08-03 00:48:07.252 [Debug] [ComfyUI-0/STDERR] got prompt
2024-08-03 00:48:07.270 [Debug] [BackendHandler] Backend request #283 for model sdxlpony/DucHaiten-Pony-Real - Pony-Real_v10.safetensors, maxWait=7.00:00:00.
2024-08-03 00:48:07.270 [Debug] [BackendHandler] Backend request #283 found correct model on #0
2024-08-03 00:48:07.270 [Debug] [BackendHandler] Backend request #283 finished.
2024-08-03 00:48:07.271 [Debug] [ComfyUI-0/STDERR] got prompt
2024-08-03 00:48:07.278 [Debug] [ComfyUI-0/STDERR] !!! Exception during processing!!! 'NoneType' object has no attribute 'lower'
2024-08-03 00:48:07.279 [Warning] [ComfyUI-0/STDERR] Traceback (most recent call last):
2024-08-03 00:48:07.279 [Warning] [ComfyUI-0/STDERR]   File "/config/07-StableSwarm/StableSwarmUI/dlbackend/ComfyUI/execution.py", line 152, in recursive_execute
2024-08-03 00:48:07.279 [Warning] [ComfyUI-0/STDERR]     output_data, output_ui = get_output_data(obj, input_data_all)
2024-08-03 00:48:07.279 [Warning] [ComfyUI-0/STDERR]                              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-08-03 00:48:07.279 [Warning] [ComfyUI-0/STDERR]   File "/config/07-StableSwarm/StableSwarmUI/dlbackend/ComfyUI/execution.py", line 82, in get_output_data
2024-08-03 00:48:07.279 [Warning] [ComfyUI-0/STDERR]     return_values = map_node_over_list(obj, input_data_all, obj.FUNCTION, allow_interrupt=True)
2024-08-03 00:48:07.279 [Warning] [ComfyUI-0/STDERR]                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-08-03 00:48:07.279 [Warning] [ComfyUI-0/STDERR]   File "/config/07-StableSwarm/StableSwarmUI/dlbackend/ComfyUI/execution.py", line 75, in map_node_over_list
2024-08-03 00:48:07.279 [Warning] [ComfyUI-0/STDERR]     results.append(getattr(obj, func)(**slice_dict(input_data_all, i)))
2024-08-03 00:48:07.279 [Warning] [ComfyUI-0/STDERR]                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-08-03 00:48:07.279 [Warning] [ComfyUI-0/STDERR]   File "/config/07-StableSwarm/StableSwarmUI/dlbackend/ComfyUI/nodes.py", line 836, in load_unet
2024-08-03 00:48:07.279 [Warning] [ComfyUI-0/STDERR]     model = comfy.sd.load_unet(unet_path, dtype=dtype)
2024-08-03 00:48:07.279 [Warning] [ComfyUI-0/STDERR]             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-08-03 00:48:07.279 [Warning] [ComfyUI-0/STDERR]   File "/config/07-StableSwarm/StableSwarmUI/dlbackend/ComfyUI/comfy/sd.py", line 621, in load_unet
2024-08-03 00:48:07.279 [Warning] [ComfyUI-0/STDERR]     sd = comfy.utils.load_torch_file(unet_path)
2024-08-03 00:48:07.279 [Warning] [ComfyUI-0/STDERR]          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-08-03 00:48:07.279 [Warning] [ComfyUI-0/STDERR]   File "/config/07-StableSwarm/StableSwarmUI/dlbackend/ComfyUI/comfy/utils.py", line 14, in load_torch_file
2024-08-03 00:48:07.279 [Warning] [ComfyUI-0/STDERR]     if ckpt.lower().endswith(".safetensors") or ckpt.lower().endswith(".sft"):
2024-08-03 00:48:07.279 [Warning] [ComfyUI-0/STDERR]        ^^^^^^^^^^
2024-08-03 00:48:07.279 [Warning] [ComfyUI-0/STDERR] AttributeError: 'NoneType' object has no attribute 'lower'
2024-08-03 00:48:07.279 [Warning] [ComfyUI-0/STDERR] 
2024-08-03 00:48:07.281 [Debug] [ComfyUI-0/STDERR] Prompt executed in 0.02 seconds
2024-08-03 00:48:07.290 [Debug] [BackendHandler] Backend request #284 for model sdxlpony/DucHaiten-Pony-Real - Pony-Real_v10.safetensors, maxWait=7.00:00:00.
2024-08-03 00:48:07.310 [Debug] [BackendHandler] Backend request #285 for model sdxlpony/DucHaiten-Pony-Real - Pony-Real_v10.safetensors, maxWait=7.00:00:00.
2024-08-03 00:48:07.330 [Debug] [BackendHandler] Backend request #286 for model sdxlpony/DucHaiten-Pony-Real - Pony-Real_v10.safetensors, maxWait=7.00:00:00.
2024-08-03 00:48:07.443 [Debug] Failed to process comfy workflow: { "5": { "inputs": { "width": 896, "height": 1152, "batch_size": 1 }, "class_type": "EmptyLatentImage" }, "6": { "inputs": { "text": "a bottle with a rainbow galaxy inside it on top of a wooden table on a table in the middle of a modern kitchen", "clip": [ "11", 0 ] }, "class_type": "CLIPTextEncode" }, "8": { "inputs": { "samples": [ "13", 0 ], "vae": [ "10", 0 ] }, "class_type": "VAEDecode" }, "9": { "inputs": { "images": [ "28", 0 ] }, "class_type": "SwarmSaveImageWS" }, "10": { "inputs": { "vae_name": "flux/ae.sft" }, "class_type": "VAELoader" }, "11": { "inputs": { "clip_name1": "t5xxl_fp8_e4m3fn.safetensors", "clip_name2": "clip_l.safetensors", "type": "flux" }, "class_type": "DualCLIPLoader" }, "12": { "inputs": { "unet_name": "sdxlpony/DucHaiten-Pony-Real - Pony-Real_v10.safetensors", "weight_dtype": "fp8_e4m3fn" }, "class_type": "UNETLoader" }, "13": { "inputs": { "noise": [ "25", 0 ], "guider": [ "22", 0 ], "sampler": [ "16", 0 ], "sigmas": [ "17", 0 ], "latent_image": [ "5", 0 ] }, "class_type": "SamplerCustomAdvanced" }, "16": { "inputs": { "sampler_name": "euler" }, "class_type": "KSamplerSelect" }, "17": { "inputs": { "scheduler": "simple", "steps": 4, "denoise": 1, "model": [ "12", 0 ] }, "class_type": "BasicScheduler" }, "22": { "inputs": { "model": [ "12", 0 ], "conditioning": [ "6", 0 ] }, "class_type": "BasicGuider" }, "25": { "inputs": { "noise_seed": 1593229728 }, "class_type": "RandomNoise" }, "27": { "inputs": { "input_faces_order": "large-small", "input_faces_index": "0", "detect_gender_input": "no", "source_faces_order": "large-small", "source_faces_index": "0", "detect_gender_source": "no", "console_log_level": 1 }, "class_type": "ReActorOptions" }, "28": { "inputs": { "enabled": True, "swap_model": "inswapper_128.onnx", "facedetection": "retinaface_resnet50", "face_restore_model": "GFPGANv1.4.pth", "face_restore_visibility": 1, "codeformer_weight": 0.75, "input_image": [ "8", 0 ], "source_image": [ "30", 0 ], "options": [ "27", 0 ], "face_boost": [ "29", 0 ] }, "class_type": "ReActorFaceSwapOpt" }, "29": { "inputs": { "enabled": True, "boost_model": "GFPGANv1.4.pth", "interpolation": "Bicubic", "visibility": 0.75, "codeformer_weight": 0.5, "restore_with_main_after": False }, "class_type": "ReActorFaceBoost" }, "30": { "inputs": { "image": "2024-07-02_21-55-33-3-587484196.png", "upload": "image" }, "class_type": "LoadImage" } } for inputs T2IParamInput(comfyworkflowparammetadata: {"comfyrawworkflowinputtextcliptextencodenodetextg":{"name":"text","default":"a bottle with a rainbow galaxy inside it on top of a wooden table on a table in the middle of a modern kitchen","id":"comfyrawworkflowinputtextcliptextencodenodetextg","type":"te..., comfyworkflowraw: {"5":{"inputs":{"width":"%%_COMFYFIXME_${width:896}_ENDFIXME_%%","height":"%%_COMFYFIXME_${height:1152}_ENDFIXME_%%","batch_size":"%%_COMFYFIXME_${batchsize:1}_ENDFIXME_%%"},"class_type":"EmptyLatentImage"},"6":{"inputs":{"text":"${comfyrawworkflowinputtex..., model: sdxlpony/DucHaiten-Pony-Real - Pony-Real_v10, images: 8, aspectratio: Custom, width: 896, height: 1152, automaticvae: True, seed: 1054168266, prompt: , negativeprompt: )
2024-08-03 00:48:07.443 [Debug] Refused to generate image for local: ComfyUI execution error: 'NoneType' object has no attribute 'lower'
2024-08-03 00:48:07.443 [Debug] [BackendHandler] Backend request #284 found correct model on #0
2024-08-03 00:48:07.443 [Debug] [BackendHandler] Backend request #284 finished.
2024-08-03 00:48:07.443 [Debug] [BackendHandler] Backend request #285 found correct model on #0
2024-08-03 00:48:07.443 [Debug] [BackendHandler] Backend request #285 finished.
2024-08-03 00:48:07.443 [Debug] [BackendHandler] Backend request #286 found correct model on #0
2024-08-03 00:48:07.443 [Debug] [BackendHandler] Backend request #286 finished.
2024-08-03 00:48:07.456 [Debug] [ComfyUI-0/STDERR] !!! Exception during processing!!! 'NoneType' object has no attribute 'lower'
2024-08-03 00:48:07.458 [Warning] [ComfyUI-0/STDERR] Traceback (most recent call last):
2024-08-03 00:48:07.458 [Warning] [ComfyUI-0/STDERR]   File "/config/07-StableSwarm/StableSwarmUI/dlbackend/ComfyUI/execution.py", line 152, in recursive_execute
2024-08-03 00:48:07.458 [Warning] [ComfyUI-0/STDERR]     output_data, output_ui = get_output_data(obj, input_data_all)
2024-08-03 00:48:07.458 [Warning] [ComfyUI-0/STDERR]                              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-08-03 00:48:07.458 [Warning] [ComfyUI-0/STDERR]   File "/config/07-StableSwarm/StableSwarmUI/dlbackend/ComfyUI/execution.py", line 82, in get_output_data
2024-08-03 00:48:07.458 [Warning] [ComfyUI-0/STDERR]     return_values = map_node_over_list(obj, input_data_all, obj.FUNCTION, allow_interrupt=True)
2024-08-03 00:48:07.458 [Warning] [ComfyUI-0/STDERR]                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-08-03 00:48:07.458 [Warning] [ComfyUI-0/STDERR]   File "/config/07-StableSwarm/StableSwarmUI/dlbackend/ComfyUI/execution.py", line 75, in map_node_over_list
2024-08-03 00:48:07.458 [Warning] [ComfyUI-0/STDERR]     results.append(getattr(obj, func)(**slice_dict(input_data_all, i)))
2024-08-03 00:48:07.458 [Warning] [ComfyUI-0/STDERR]                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-08-03 00:48:07.458 [Warning] [ComfyUI-0/STDERR]   File "/config/07-StableSwarm/StableSwarmUI/dlbackend/ComfyUI/nodes.py", line 836, in load_unet
2024-08-03 00:48:07.458 [Warning] [ComfyUI-0/STDERR]     model = comfy.sd.load_unet(unet_path, dtype=dtype)
2024-08-03 00:48:07.458 [Warning] [ComfyUI-0/STDERR]             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-08-03 00:48:07.458 [Warning] [ComfyUI-0/STDERR]   File "/config/07-StableSwarm/StableSwarmUI/dlbackend/ComfyUI/comfy/sd.py", line 621, in load_unet
2024-08-03 00:48:07.458 [Warning] [ComfyUI-0/STDERR]     sd = comfy.utils.load_torch_file(unet_path)
2024-08-03 00:48:07.458 [Warning] [ComfyUI-0/STDERR]          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-08-03 00:48:07.458 [Warning] [ComfyUI-0/STDERR]   File "/config/07-StableSwarm/StableSwarmUI/dlbackend/ComfyUI/comfy/utils.py", line 14, in load_torch_file
2024-08-03 00:48:07.458 [Warning] [ComfyUI-0/STDERR]     if ckpt.lower().endswith(".safetensors") or ckpt.lower().endswith(".sft"):
2024-08-03 00:48:07.458 [Warning] [ComfyUI-0/STDERR]        ^^^^^^^^^^
2024-08-03 00:48:07.458 [Warning] [ComfyUI-0/STDERR] AttributeError: 'NoneType' object has no attribute 'lower'
2024-08-03 00:48:07.458 [Warning] [ComfyUI-0/STDERR] 
2024-08-03 00:48:07.460 [Debug] [ComfyUI-0/STDERR] Prompt executed in 0.02 seconds
2024-08-03 00:48:07.461 [Debug] Failed to process comfy workflow: { "5": { "inputs": { "width": 896, "height": 1152, "batch_size": 1 }, "class_type": "EmptyLatentImage" }, "6": { "inputs": { "text": "a bottle with a rainbow galaxy inside it on top of a wooden table on a table in the middle of a modern kitchen", "clip": [ "11", 0 ] }, "class_type": "CLIPTextEncode" }, "8": { "inputs": { "samples": [ "13", 0 ], "vae": [ "10", 0 ] }, "class_type": "VAEDecode" }, "9": { "inputs": { "images": [ "28", 0 ] }, "class_type": "SwarmSaveImageWS" }, "10": { "inputs": { "vae_name": "flux/ae.sft" }, "class_type": "VAELoader" }, "11": { "inputs": { "clip_name1": "t5xxl_fp8_e4m3fn.safetensors", "clip_name2": "clip_l.safetensors", "type": "flux" }, "class_type": "DualCLIPLoader" }, "12": { "inputs": { "unet_name": "sdxlpony/DucHaiten-Pony-Real - Pony-Real_v10.safetensors", "weight_dtype": "fp8_e4m3fn" }, "class_type": "UNETLoader" }, "13": { "inputs": { "noise": [ "25", 0 ], "guider": [ "22", 0 ], "sampler": [ "16", 0 ], "sigmas": [ "17", 0 ], "latent_image": [ "5", 0 ] }, "class_type": "SamplerCustomAdvanced" }, "16": { "inputs": { "sampler_name": "euler" }, "class_type": "KSamplerSelect" }, "17": { "inputs": { "scheduler": "simple", "steps": 4, "denoise": 1, "model": [ "12", 0 ] }, "class_type": "BasicScheduler" }, "22": { "inputs": { "model": [ "12", 0 ], "conditioning": [ "6", 0 ] }, "class_type": "BasicGuider" }, "25": { "inputs": { "noise_seed": 611187908 }, "class_type": "RandomNoise" }, "27": { "inputs": { "input_faces_order": "large-small", "input_faces_index": "0", "detect_gender_input": "no", "source_faces_order": "large-small", "source_faces_index": "0", "detect_gender_source": "no", "console_log_level": 1 }, "class_type": "ReActorOptions" }, "28": { "inputs": { "enabled": True, "swap_model": "inswapper_128.onnx", "facedetection": "retinaface_resnet50", "face_restore_model": "GFPGANv1.4.pth", "face_restore_visibility": 1, "codeformer_weight": 0.75, "input_image": [ "8", 0 ], "source_image": [ "30", 0 ], "options": [ "27", 0 ], "face_boost": [ "29", 0 ] }, "class_type": "ReActorFaceSwapOpt" }, "29": { "inputs": { "enabled": True, "boost_model": "GFPGANv1.4.pth", "interpolation": "Bicubic", "visibility": 0.75, "codeformer_weight": 0.5, "restore_with_main_after": False }, "class_type": "ReActorFaceBoost" }, "30": { "inputs": { "image": "2024-07-02_21-55-33-3-587484196.png", "upload": "image" }, "class_type": "LoadImage" } } for inputs T2IParamInput(comfyworkflowparammetadata: {"comfyrawworkflowinputtextcliptextencodenodetextg":{"name":"text","default":"a bottle with a rainbow galaxy inside it on top of a wooden table on a table in the middle of a modern kitchen","id":"comfyrawworkflowinputtextcliptextencodenodetextg","type":"te..., comfyworkflowraw: {"5":{"inputs":{"width":"%%_COMFYFIXME_${width:896}_ENDFIXME_%%","height":"%%_COMFYFIXME_${height:1152}_ENDFIXME_%%","batch_size":"%%_COMFYFIXME_${batchsize:1}_ENDFIXME_%%"},"class_type":"EmptyLatentImage"},"6":{"inputs":{"text":"${comfyrawworkflowinputtex..., model: sdxlpony/DucHaiten-Pony-Real - Pony-Real_v10, images: 8, aspectratio: Custom, width: 896, height: 1152, automaticvae: True, seed: 1054168267, prompt: , negativeprompt: )
2024-08-03 00:48:07.462 [Debug] Refused to generate image for local: ComfyUI execution error: 'NoneType' object has no attribute 'lower'

In Swarm Generate tab, I went to where it says '(Custom Comfy Workflow ) Model' and set the model to '(None)'. Cos... I dunno what I'm doing, really, so I thought I'd try it anyway. It was set to that Pony model before I did this.

Then I got this error:

2024-08-03 00:50:38.220 [Debug] Ignoring input to parameter model because the value maps to null.
2024-08-03 00:50:38.220 [Info] User local requested 8 images with model ''...
2024-08-03 00:50:38.221 [Debug] [BackendHandler] Backend request #287 for model any, maxWait=7.00:00:00.
2024-08-03 00:50:38.221 [Debug] [BackendHandler] Backend request #287 will claim #0
2024-08-03 00:50:38.221 [Debug] [BackendHandler] Backend request #287 finished.
2024-08-03 00:50:38.241 [Debug] [BackendHandler] Backend request #288 for model any, maxWait=7.00:00:00.
2024-08-03 00:50:38.261 [Debug] [BackendHandler] Backend request #289 for model any, maxWait=7.00:00:00.
2024-08-03 00:50:38.281 [Debug] [BackendHandler] Backend request #290 for model any, maxWait=7.00:00:00.
2024-08-03 00:50:38.301 [Debug] [BackendHandler] Backend request #291 for model any, maxWait=7.00:00:00.
2024-08-03 00:50:38.302 [Error] Internal error processing T2I request: System.NullReferenceException: Object reference not set to an instance of an object.
   at SwarmUI.Builtin_ComfyUIBackend.ComfyUIAPIAbstractBackend.<>c__DisplayClass22_0.<CreateWorkflow>b__2(String tag) in /config/07-StableSwarm/StableSwarmUI/src/BuiltinExtensions/ComfyUIBackend/ComfyUIAPIAbstractBackend.cs:line 642
   at FreneticUtilities.FreneticToolkit.StringConversionHelper.QuickSimpleTagFiller(String taggedText, String prefix, String suffix, Func`2 tagReader, Boolean doSubtags, Int32 maxRecurse)
   at SwarmUI.Builtin_ComfyUIBackend.ComfyUIAPIAbstractBackend.CreateWorkflow(T2IParamInput user_input, Func`2 initImageFixer, String ModelFolderFormat, HashSet`1 features) in /config/07-StableSwarm/StableSwarmUI/src/BuiltinExtensions/ComfyUIBackend/ComfyUIAPIAbstractBackend.cs:line 575
   at SwarmUI.Builtin_ComfyUIBackend.ComfyUIAPIAbstractBackend.GenerateLive(T2IParamInput user_input, String batchId, Action`1 takeOutput) in /config/07-StableSwarm/StableSwarmUI/src/BuiltinExtensions/ComfyUIBackend/ComfyUIAPIAbstractBackend.cs:line 732
   at SwarmUI.Text2Image.T2IEngine.CreateImageTask(T2IParamInput user_input, String batchId, GenClaim claim, Action`1 output, Action`1 setError, Boolean isWS, Single backendTimeoutMin, Action`2 saveImages, Boolean canCallTools) in /config/07-StableSwarm/StableSwarmUI/src/Text2Image/T2IEngine.cs:line 259
2024-08-03 00:50:38.302 [Debug] Refused to generate image for local: Something went wrong while generating images.
2024-08-03 00:50:38.302 [Debug] [BackendHandler] Backend request #288 will claim #0
2024-08-03 00:50:38.302 [Debug] [BackendHandler] Backend request #288 finished.
2024-08-03 00:50:38.302 [Debug] [BackendHandler] Backend request #289 will claim #0
2024-08-03 00:50:38.302 [Debug] [BackendHandler] Backend request #289 finished.
2024-08-03 00:50:38.302 [Debug] [BackendHandler] Backend request #290 will claim #0
2024-08-03 00:50:38.302 [Debug] [BackendHandler] Backend request #290 finished.
2024-08-03 00:50:38.302 [Debug] [BackendHandler] Backend request #291 will claim #0
2024-08-03 00:50:38.302 [Debug] [BackendHandler] Backend request #291 finished.

But also, where you load the models at the bottom, I searched that list for Flux and it just isn't there. I've definitely restarted the whole thing - not just Comfy.

For some reason, Swarm just isn't seeing Flux as an available model for me in that particular dropdown even though the workflow does, indeed, load it and run it just fine.

mcmonkey4eva commented 1 month ago

make sure your Swarm install is up to date, Flux support was naturally just added yesterday

Taruvi commented 1 month ago

Every time I start it, it automatically updates. It's a feature of this Docker build I'm using.

Speaking of which...

I might be on to something here. This Docker build has a bunch of front ends that have a common shared models directory to avoid the duplication of models and the obvious huge space waste that would cause. So generally I put models, etc. into the shared directory but sometimes that doesn't work and I have to do something a bit more bespoke to that UI.

So what's happening here is I tried using the shared directory, as I always do first. When that doesn't work, I have to dig down and find out where the model will be accepted and seen by the UI.

So that's where I currently am now: Stopping the container, moving the models around and starting the container again to see if it will take this time.

Just got this error:

2024-08-03 01:26:39.512 [Debug] Refused to generate image for local: ComfyUI execution error: 'NoneType' object has no attribute 'lower'
2024-08-03 01:26:39.512 [Debug] Refused to generate image for local: ComfyUI execution error: 'NoneType' object has no attribute 'lower'
2024-08-03 01:26:39.513 [Debug] [BackendHandler] Backend request #9 found correct model on #0
2024-08-03 01:26:39.513 [Debug] [BackendHandler] Backend request #9 finished.
2024-08-03 01:26:39.513 [Debug] [BackendHandler] Backend request #10 found correct model on #0
2024-08-03 01:26:39.513 [Debug] [BackendHandler] Backend request #10 finished.
2024-08-03 01:26:39.513 [Debug] [BackendHandler] Backend request #11 found correct model on #0
2024-08-03 01:26:39.513 [Debug] [BackendHandler] Backend request #11 finished.
2024-08-03 01:26:50.443 [Info] User local requested 8 images with model 'flux1-schnell.sft'...
2024-08-03 01:26:50.444 [Debug] [BackendHandler] Backend request #12 for model flux1-schnell.sft, maxWait=7.00:00:00.
2024-08-03 01:26:50.444 [Debug] [BackendHandler] backend #0 will load a model: flux1-schnell.sft, with 1 requests waiting for 0 seconds
2024-08-03 01:26:50.574 [Debug] [BackendHandler] Backend request #13 for model flux1-schnell.sft, maxWait=7.00:00:00.
2024-08-03 01:26:50.577 [Info] Downloading clip_l_sdxl_base.safetensors to Models/clip/clip_l_sdxl_base.safetensors...
2024-08-03 01:26:50.594 [Debug] [BackendHandler] Backend request #14 for model flux1-schnell.sft, maxWait=7.00:00:00.
2024-08-03 01:26:50.614 [Debug] [BackendHandler] Backend request #15 for model flux1-schnell.sft, maxWait=7.00:00:00.
2024-08-03 01:26:50.634 [Debug] [BackendHandler] Backend request #16 for model flux1-schnell.sft, maxWait=7.00:00:00.
2024-08-03 01:26:52.316 [Info] clip_l_sdxl_base.safetensors download at 34.1%...
2024-08-03 01:26:53.417 [Info] clip_l_sdxl_base.safetensors download at 74.7%...
2024-08-03 01:26:53.919 [Info] clip_l_sdxl_base.safetensors download at 100.0%...
2024-08-03 01:26:53.919 [Info] Downloading complete, continuing.
2024-08-03 01:26:53.921 [Info] Downloading t5xxl_enconly.safetensors to Models/clip/t5xxl_enconly.safetensors...
2024-08-03 01:27:05.377 [Info] t5xxl_enconly.safetensors download at 5.8%...
2024-08-03 01:27:07.478 [Info] t5xxl_enconly.safetensors download at 10.1%...
2024-08-03 01:27:10.480 [Info] t5xxl_enconly.safetensors download at 16.4%...
2024-08-03 01:27:12.981 [Info] t5xxl_enconly.safetensors download at 20.6%...
2024-08-03 01:27:16.084 [Info] t5xxl_enconly.safetensors download at 26.9%...
2024-08-03 01:27:18.086 [Info] t5xxl_enconly.safetensors download at 31.1%...
2024-08-03 01:27:20.188 [Info] t5xxl_enconly.safetensors download at 35.4%...
2024-08-03 01:27:23.292 [Info] t5xxl_enconly.safetensors download at 41.4%...
2024-08-03 01:27:25.393 [Info] t5xxl_enconly.safetensors download at 45.7%...
2024-08-03 01:27:28.496 [Info] t5xxl_enconly.safetensors download at 51.9%...
2024-08-03 01:27:30.500 [Info] t5xxl_enconly.safetensors download at 56.2%...
2024-08-03 01:27:32.502 [Info] t5xxl_enconly.safetensors download at 60.4%...
2024-08-03 01:27:35.606 [Info] t5xxl_enconly.safetensors download at 65.9%...
2024-08-03 01:27:37.707 [Info] t5xxl_enconly.safetensors download at 70.0%...
2024-08-03 01:27:40.709 [Info] t5xxl_enconly.safetensors download at 76.3%...
2024-08-03 01:27:42.811 [Info] t5xxl_enconly.safetensors download at 80.6%...
2024-08-03 01:27:45.913 [Info] t5xxl_enconly.safetensors download at 86.6%...
2024-08-03 01:27:48.014 [Info] t5xxl_enconly.safetensors download at 90.7%...
2024-08-03 01:27:50.318 [Info] t5xxl_enconly.safetensors download at 95.6%...
2024-08-03 01:27:52.422 [Info] t5xxl_enconly.safetensors download at 100.0%...
2024-08-03 01:27:52.422 [Info] Downloading complete, continuing.
2024-08-03 01:27:52.423 [Error] [BackendHandler] backend #0 failed to load model with error: No default Flux VAE found, please download an Flux VAE and set it as default in User Settings
2024-08-03 01:27:52.423 [Warning] [BackendHandler] backend #0 failed to load model flux1-schnell.sft

That 't5xxl_enconly.safetensors' is something I've never seen before. It decides to download now...? Even though it was in a sense working perfectly fine before? OK...

Also, I now see the Flux model in the dropdown. So I'm getting there. I just need to sort out the VAE and I think that might be it.

Any other checkpoints just work with the shared directory but Flux is obviously something a bit different.

Taruvi commented 1 month ago

OK, finally got everything where they need to be in order to be recognised. I can generate and it lists Flux as the model. Everything's sorted. Also, I don't need to add that extra primitive. So all is good there.

In the process of doing this, I found a very minor issue:

flux-default-vae

I have two versions of 'None'. I mean, it's working but... Well, this is a thing.

I also have two versions of 'None' in the SDXL VAE as well, which is the only other VAE (I have two different ones) I have downloaded.

It may be my funky Docker directory setup or it may be something else. Normally all these directories are fully transparent to the UI but today everything decided to have a hissy fit instead.

mcmonkey4eva commented 1 month ago

Oh, nope, that was wonked in the code from the Flux update - my bad, fixed.

mcmonkeyprojects / SwarmUI