comfyanonymous / ComfyUI

The most powerful and modular diffusion model GUI, api and backend with a graph/nodes interface.
https://www.comfy.org/
GNU General Public License v3.0
49.96k stars 5.26k forks source link

SFT and FLux not recognized even after updating #4179

Open crystalwizard opened 1 month ago

crystalwizard commented 1 month ago

Expected Behavior

updated to the latest version of comfy with the support for flux. downloaded the example workflows and flux ae.sft and models to the specified folders, restarted comfy, expected to be able to run flux models with the example workflows

Actual Behavior

image instead, comfy can't recognize flux as the type, only sdxl and sd3 - and doesn't know what to do with ae.sft

Steps to Reproduce

use an older version of comfy, run update with manager, and then restart and try to run flux

Debug Logs

To see the GUI go to: http://127.0.0.1:8188
FETCH DATA from: C:\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-Manager\extension-node-map.json [DONE]
got prompt
Failed to validate prompt for output 9:
* UNETLoader 12:
  - Value not in list: unet_name: 'flux1-dev.sft' not in ['xl-inpaint-0.1\\diffusion_pytorch_model.fp16.safetensors', 'xl-inpaint-0.1\\diffusion_pytorch_model.safetensors']
* DualCLIPLoader 11:
  - Value not in list: type: 'flux' not in ['sdxl', 'sd3']
* VAELoader 10:
  - Value not in list: vae_name: 'ae.sft' not in ['kl-f8-anime2.ckpt', 'orangemix.vae.pt', 'sdxl.vae.safetensors', 'sdxl_vae.safetensors', 'vae-ft-mse-840000-ema-pruned.ckpt', 'vae-ft-mse-840000-ema-pruned.safetensors', 'taesd', 'taesdxl']
Output will be ignored
invalid prompt: {'type': 'prompt_outputs_failed_validation', 'message': 'Prompt outputs failed validation', 'details': '', 'extra_info': {}}

Other

No response

Raigor2 commented 1 month ago

any updates? same issue

silentoplayz commented 1 month ago

Same here, although I am using ComfyUI-Zluda. Edit: I fixed my problem by re-cloning from scratch and them copy/pasting my models folder over into the new cloned version.

bdc8408 commented 1 month ago

Encountering the same issue here. Same steps. Did a fresh install of comfyui and am running off the portable windows GPU script.

kakaxixx commented 1 month ago

same issue

Creative-comfyUI commented 1 month ago

First you have to update to pytorch 2.4 . Second Update ComfyUI Third all the sft file must be rename to safetensors. IMPORTANT : if you are on Mac M, it is better to quit all applications, restart comfyUI in terminal, open your browser and load the Flux workflow. This way you will have less problems. 16MG of RAM minimum is required and enough space on your disk. Unfortunatly it seems that if you don't have 32 MG of RAM the application will always crash even with the small UNET model and the fp8 dualclip.

comfyanonymous commented 1 month ago

If you have this issue it means you need to properly update ComfyUI. If you are on the standalone you should run: update/update_comfyui.bat

ltdrdata commented 1 month ago

dont update from manager, it aint working often

Typically, this phenomenon occurs when the user has modified some of the ComfyUI files. (The most common case is when parts of folders are replaced with symbolic links. This has spread as a tip for sharing model directories.) For safety reasons, ComfyUI-Manager takes a conservative update policy when conflicts occur. In this case, the update fails.

crystalwizard commented 1 month ago

I haven't modified, or even touched, the comfyUI files at all, but i did do the update via manager

crystalwizard commented 1 month ago

First you have to update to pytorch 2.4 . Second Update ComfyUI Third all the sft file must be rename to safetensors. IMPORTANT : if you are on Mac M, it is better to quit all applications, restart comfyUI in terminal, open your browser and load the Flux workflow. This way you will have less problems. 16MG of RAM minimum is required and enough space on your disk. Unfortunatly it seems that if you don't have 32 MG of RAM the application will always crash even with the small UNET model and the fp8 dualclip.

Why didn't updating comfy via Manager work?

Creative-comfyUI commented 1 month ago

Why didn't updating comfy via Manager work?

Have you updated pytorch to 2.4 ? Even if you update comfyUI, if you don't update pytorch, the problem will persist.

sombralabs commented 1 month ago

Update to pytorch 2.4 in a venv and it should work fine.

Xeddar commented 1 month ago

Running update_comfyui_and_python_dependencies.bat in the update folder will force a full reinstall to torch 2.4. The Manager or update_comfyui.bat won't.

Amit30swgoh commented 1 month ago

ERROR: Could not detect model type of: /content/drive/MyDrive/ComfyUI/models/checkpoints/Flux/flux1-dev-fp8.safetensors

image

ltdrdata commented 1 month ago

ERROR: Could not detect model type of: /content/drive/MyDrive/ComfyUI/models/checkpoints/Flux/flux1-dev-fp8.safetensors

image

Because it's not a checkpoint model. You have to move it to models/unet. And use Load Diffusion Model instead of CheckpointLoaderSimple.

Darkenlord1 commented 1 month ago

ERROR: Could not detect model type of: /content/drive/MyDrive/ComfyUI/models/checkpoints/Flux/flux1-dev-fp8.safetensors image

Because it's not a checkpoint model. You have to move it to models/unet. And use Load Diffusion Model instead of CheckpointLoaderSimple.

But there in included workflow it is literally CheckpointLoaderSimple - https://comfyanonymous.github.io/ComfyUI_examples/flux/#simple-to-use-fp8-checkpoint-version

Also experiensing the same problem:

Error occurred when executing CheckpointLoaderSimple:

ERROR: Could not detect model type of: D:\SD\WebUI 1.10.0-RC\models/Stable-diffusion\unet\flux1-dev-fp8.safetensors

File "D:\SD\ComfyUI_windows_portable\ComfyUI\execution.py", line 152, in recursive_execute
output_data, output_ui = get_output_data(obj, input_data_all)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "D:\SD\ComfyUI_windows_portable\ComfyUI\execution.py", line 82, in get_output_data
return_values = map_node_over_list(obj, input_data_all, obj.FUNCTION, allow_interrupt=True)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "D:\SD\ComfyUI_windows_portable\ComfyUI\execution.py", line 75, in map_node_over_list
results.append(getattr(obj, func)(**slice_dict(input_data_all, i)))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "D:\SD\ComfyUI_windows_portable\ComfyUI\nodes.py", line 518, in load_checkpoint
out = comfy.sd.load_checkpoint_guess_config(ckpt_path, output_vae=True, output_clip=True, embedding_directory=folder_paths.get_folder_paths("embeddings"))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "D:\SD\ComfyUI_windows_portable\ComfyUI\comfy\sd.py", line 518, in load_checkpoint_guess_config
raise RuntimeError("ERROR: Could not detect model type of: {}".format(ckpt_path))
ltdrdata commented 1 month ago

ERROR: Could not detect model type of: /content/drive/MyDrive/ComfyUI/models/checkpoints/Flux/flux1-dev-fp8.safetensors image

Because it's not a checkpoint model. You have to move it to models/unet. And use Load Diffusion Model instead of CheckpointLoaderSimple.

But there in included workflow it is literally CheckpointLoaderSimple - https://comfyanonymous.github.io/ComfyUI_examples/flux/#simple-to-use-fp8-checkpoint-version

Also experiensing the same problem:

Error occurred when executing CheckpointLoaderSimple:

ERROR: Could not detect model type of: D:\SD\WebUI 1.10.0-RC\models/Stable-diffusion\unet\flux1-dev-fp8.safetensors

File "D:\SD\ComfyUI_windows_portable\ComfyUI\execution.py", line 152, in recursive_execute
output_data, output_ui = get_output_data(obj, input_data_all)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "D:\SD\ComfyUI_windows_portable\ComfyUI\execution.py", line 82, in get_output_data
return_values = map_node_over_list(obj, input_data_all, obj.FUNCTION, allow_interrupt=True)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "D:\SD\ComfyUI_windows_portable\ComfyUI\execution.py", line 75, in map_node_over_list
results.append(getattr(obj, func)(**slice_dict(input_data_all, i)))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "D:\SD\ComfyUI_windows_portable\ComfyUI\nodes.py", line 518, in load_checkpoint
out = comfy.sd.load_checkpoint_guess_config(ckpt_path, output_vae=True, output_clip=True, embedding_directory=folder_paths.get_folder_paths("embeddings"))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "D:\SD\ComfyUI_windows_portable\ComfyUI\comfy\sd.py", line 518, in load_checkpoint_guess_config
raise RuntimeError("ERROR: Could not detect model type of: {}".format(ckpt_path))

What's important is the URL from which you downloaded that model. If what you received is the 11GB model distributed by kijai, that's a diffusion model. If it's the 17GB model distributed by Comfy Org, that's a checkpoint model. Both have the same name.

crystalwizard commented 1 month ago

Why didn't updating comfy via Manager work?

Have you updated pytorch to 2.4 ? Even if you update comfyUI, if you don't update pytorch, the problem will persist.

i updated torch after you said to, but then Cocktail Peanut said that he was running on a lower level of torch just fine. he just put flux into pinioko. so i don't think it's the level of torch, unless ComfyUI is requiring the higher level?

1990-s commented 1 month ago

I think there may be a problem with the update py file, you can try to download a new release and overwrite the update folder of the current version with the latest update, the problem was solved after I did this!

2PeteShakur commented 1 month ago

i had this issue after updating the portable version and it was fixed by renaming .sft to .safetensors - worked for me, now flux can see the vae and select and use it. strange, cos i have it working fine on another system and it sees .sft without issue (altho it wasn't the portable version) - anyway it worked for me as described, thanks!

Darkenlord1 commented 1 month ago

What's important is the URL from which you downloaded that model. If what you received is the 11GB model distributed by kijai, that's a diffusion model. If it's the 17GB model distributed by Comfy Org, that's a checkpoint model. Both have the same name.

That was the issue, thanks!

Shaburu commented 2 weeks ago

Anyone expieriencing this issue when setting up flux in the comfyui workflow?

` Error occurred when executing UNETLoader:

'conv_in.weight'

File "C:\Users\HP\Desktop\Code\ComfyUI_windows_portable\ComfyUI\execution.py", line 151, in recursive_execute for k, v in inputs.items(): ^^^ File "C:\Users\HP\Desktop\Code\ComfyUI_windows_portable\ComfyUI\execution.py", line 81, in get_output_data self.objects = HierarchicalCache(CacheKeySetID) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\HP\Desktop\Code\ComfyUI_windows_portable\ComfyUI\execution.py", line 74, in map_node_over_list self.ui = LRUCache(CacheKeySetInputSignature, max_size=cache_size) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\HP\Desktop\Code\ComfyUI_windows_portable\ComfyUI\nodes.py", line 829, in load_unet File "C:\Users\HP\Desktop\Code\ComfyUI_windows_portable\ComfyUI\comfy\sd.py", line 600, in load_unet if inital_load_device != torch.device("cpu"): ^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\HP\Desktop\Code\ComfyUI_windows_portable\ComfyUI\comfy\sd.py", line 573, in load_unet_state_dict File "C:\Users\HP\Desktop\Code\ComfyUI_windows_portable\ComfyUI\comfy\model_detection.py", line 447, in model_config_from_diffusers_unet 'num_classes': 'sequential', 'adm_in_channels': 2816, 'dtype': dtype, 'in_channels': 4, 'model_channels': 320, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\HP\Desktop\Code\ComfyUI_windows_portable\ComfyUI\comfy\model_detection.py", line 329, in unet_config_from_diffusers_unet t_out = [] `

Shaburu commented 2 weeks ago

Workflow image Note: My Flux is not showing up as well, could it be i need to re-download the thing from scratch? Or maybe i haven't downloaded the Flux thing? What am i missing? any help is appreciated! Thanks image

ltdrdata commented 2 weeks ago

Workflow image Note: My Flux is not showing up as well, could it be i need to re-download the thing from scratch? Or maybe i haven't downloaded the Flux thing? What am i missing? any help is appreciated! Thanks image

Your ComfyUI is outdated version. In the latest version, UNetLoader has been renamed to Load Diffusion Model. If the node name in your workflow hasn't changed, it's still incorrectly installed/updated, so please double-check the installation/update method.