Efficient Loader for SD3

[a-l-e-x-d-s-9]

I wanted to test SD3 training results with a grid made with Efficient Loader with other custom nodes that use Efficient Loader. Unfortunately, there are a few issues with the SD3 Efficient Loader. It only loads a single model - no way to combine different TE, VAE, DiT. With SD3 training, the trained DiT is given in a diffusers format that can be loaded with UnetLoader - and there is no way to use it with Efficient Loader.

[jags111]

i will have to make a node for SD3 from this and waiting on same so it can load easy.

[jags111]

here is the test run using SD 3 and eff sampler. May be this will help your needs.

incase you need something else can you share your workflow so i can advise what needs to be done.

update or send feedback

[alex13by]

What is missing is a loader for SD3, like an efficiency loader for SDXL

[a-l-e-x-d-s-9]

Here is a workflow that requires Efficient Loader due to dependencies used for XY grid, not just KSampler: Image with the workflow: Using SD3 will throw an exception (attached below). Another issue with Efficient Loader - it wants to load the whole model as a single file - and with SD3 you usually need to compound TEs from separate files. Additional issue, I'm using DiT - model transformer in diffusers format from finetuning, I can load it now only with Unet Loader node, Efficient Loader doesn't support the option to use model from Unet Loader for SD3. Exception:

Error occurred when executing Efficient Loader:

'g'

File "/home/alexds9/Documents/stable_diffusion/ComfyUI/execution.py", line 151, in recursive_execute
output_data, output_ui = get_output_data(obj, input_data_all)
File "/home/alexds9/Documents/stable_diffusion/ComfyUI/execution.py", line 81, in get_output_data
return_values = map_node_over_list(obj, input_data_all, obj.FUNCTION, allow_interrupt=True)
File "/home/alexds9/Documents/stable_diffusion/ComfyUI/execution.py", line 74, in map_node_over_list
results.append(getattr(obj, func)(**slice_dict(input_data_all, i)))
File "/home/alexds9/Documents/stable_diffusion/ComfyUI/custom_nodes/efficiency-nodes-comfyui/efficiency_nodes.py", line 179, in efficientloader
encode_prompts(positive, negative, token_normalization, weight_interpretation, clip, clip_skip,
File "/home/alexds9/Documents/stable_diffusion/ComfyUI/custom_nodes/efficiency-nodes-comfyui/efficiency_nodes.py", line 80, in encode_prompts
positive_encoded = bnk_adv_encode.AdvancedCLIPTextEncode().encode(clip, positive_prompt, token_normalization, weight_interpretation)[0]
File "/home/alexds9/Documents/stable_diffusion/ComfyUI/custom_nodes/efficiency-nodes-comfyui/py/bnk_adv_encode.py", line 312, in encode
embeddings_final, pooled = advanced_encode(clip, text, token_normalization, weight_interpretation, w_max=1.0,
File "/home/alexds9/Documents/stable_diffusion/ComfyUI/custom_nodes/efficiency-nodes-comfyui/py/bnk_adv_encode.py", line 262, in advanced_encode
return advanced_encode_from_tokens(tokenized['l'],
File "/home/alexds9/Documents/stable_diffusion/ComfyUI/custom_nodes/efficiency-nodes-comfyui/py/bnk_adv_encode.py", line 183, in advanced_encode_from_tokens
weighted_emb, pooled_base = encode_func(weighted_tokens)
File "/home/alexds9/Documents/stable_diffusion/ComfyUI/custom_nodes/efficiency-nodes-comfyui/py/bnk_adv_encode.py", line 265, in
lambda x: (clip.encode_from_tokens({'l': x}), None),
File "/home/alexds9/Documents/stable_diffusion/ComfyUI/comfy/sd.py", line 143, in encode_from_tokens
cond, pooled = self.cond_stage_model.encode_token_weights(tokens)
File "/home/alexds9/Documents/stable_diffusion/ComfyUI/comfy/sd3_clip.py", line 96, in encode_token_weights
token_weight_pairs_g = token_weight_pairs["g"]

[jags111]

okay got it.. Need to make a separate node for the sd3 component and Unet loading. current one is not designed for SD3 components and all other types of new diffuser loading. will make a new one and add same.

[jinxishe]

Yes! to use the x/y plot, it will be helpful with a loader for SD3, and if possible, please also make it for PixArt Sigma.