[Bug]: ComfyUI-GGUF custom nodes failed, thus cannot use FLUX

orangpelupa commented 3 days ago

Checklist

[X] The issue has not been resolved by following the troubleshooting guide
[X] The issue exists on a clean installation of Fooocus
[X] The issue exists in the current version of Fooocus
[X] The issue has not been reported before recently
[ ] The issue has been reported before but has not been fixed yet

What happened?

flux requires ComfyUI-GGUF, but that custom node shows as failed in the console. so i cant generate with FLUX.

generations with SDXL works fine.

Steps to reproduce the problem

install simplesdxl
generate with flux

What should have happened?

it should be generating image with flux

What browsers do you use to access Fooocus?

Microsoft Edge

Where are you running Fooocus?

Locally

What operating system are you using?

Windows 11

Console logs

[System ARGV] ['B:\\tools\\StabilityMatrix-win-x64\\Data\\Packages\\SimpleSDXL\\launch.py', '--language', 'en']
Python 3.10.11 (tags/v3.10.11:7d4cc5a, Apr  5 2023, 00:38:17) [MSC v.1929 64 bit (AMD64)]
Fooocus version: 2.5.5
Comfy version: 1dba801
SimpleSDXL version: v20240916.fc97754
Checking ...
[SimpleAI] GPU: NVIDIA GeForce RTX 4070 Ti SUPER, RAM: 32694MB, SWAP: 16000MB, VRAM: 16376MB, DiskFree: 149725MB, CUDA: 12.7
[SimpleAI] local_did/本地身份ID: Legky79Cx7V9KtDRUjy8AYExuBi
Load config data from B:\tools\StabilityMatrix-win-x64\Data\Packages\SimpleSDXL\config.txt.
Loaded preset: B:\tools\StabilityMatrix-win-x64\Data\Packages\SimpleSDXL\presets\default.json
The path_models_root: B:\tools\StabilityMatrix-win-x64\Data\Packages\SimpleSDXL\models
The path_output: B:\tools\StabilityMatrix-win-x64\Data\Packages\SimpleSDXL\outputs
Total VRAM 16376 MB, total RAM 32694 MB
Set vram state to: NORMAL_VRAM
Always offload VRAM
Device: cuda:0 NVIDIA GeForce RTX 4070 Ti SUPER : native
VAE dtype: torch.bfloat16
[Cleanup] Attempting to delete content of temp dir C:\Users\8492n\AppData\Local\Temp\fooocus
[Cleanup] Cleanup successful
Checking file is not exists / 检测到文件有缺失: B:\tools\StabilityMatrix-win-x64\Data\Packages\SimpleSDXL\models\upscale_models\fooocus_upscaler_s409985e5.bin
[SimpleAI] 程序所需基础模型包有检测异常，未完全正确安装。请检查并正确安装后，再启动程序。
Env_ready_code: 2
You do not have [juggernautXL_juggXIByRundiffusion.safetensors] but you have [juggernautXL_v9Rundiffusionphoto2.safetensors].
Fooocus will use [juggernautXL_v9Rundiffusionphoto2.safetensors] to avoid downloading new models, but you are not using the latest models.
Use --always-download-new-model to avoid fallback and always get new models.
Using pytorch cross attention
Refiner unloaded.
model_type EPS
UNet ADM Dimension 2816
Using pytorch attention in VAE
Working with z of shape (1, 4, 32, 32) = 4096 dimensions.
Using pytorch attention in VAE
[Wildcards] Refresh and Load 27/27 wildcards: animal, bodytypes, buildings, chairs, color, containers, coverings, flora, flower, focus, foods, hairstyles, jobs, materials, minilocations, moods, nationality, objects, objectstohold, outfits, patterns, poses, rpg, shotsizes, space, timeperiods, vehicles.
[OneButtonPrompt] The customized module:OneButtonPrompt is initializing ...
Running on local URL:  http://192.168.66.162:8186

To create a public link, set `share=True` in `launch()`.
extra {'cond_stage_model.clip_g.logit_scale', 'cond_stage_model.clip_l.text_projection', 'cond_stage_model.clip_l.logit_scale'}
IMPORTANT: You are using gradio version 3.41.2, however version 4.44.1 is available, please upgrade.
--------
Base model loaded: B:\tools\StabilityMatrix-win-x64\Data\Models\StableDiffusion\juggernautXL_v9Rundiffusionphoto2.safetensors
VAE loaded: None
Request to load LoRAs [('sd_xl_offset_example-lora_1.0.safetensors', 0.1)] for model [B:\tools\StabilityMatrix-win-x64\Data\Models\StableDiffusion\juggernautXL_v9Rundiffusionphoto2.safetensors].
Loaded LoRA [B:\tools\StabilityMatrix-win-x64\Data\Models\Lora\sd_xl_offset_example-lora_1.0.safetensors] for UNet [B:\tools\StabilityMatrix-win-x64\Data\Models\StableDiffusion\juggernautXL_v9Rundiffusionphoto2.safetensors] with 788 keys at weight 0.1.
Fooocus V2 Expansion: Vocab with 642 words.
Fooocus Expansion engine loaded for cuda:0, use_fp16 = True.
Requested to load SDXLClipModel
Requested to load GPT2LMHeadModel
Loading 2 new models
[Fooocus Model Management] Moving model(s) has taken 0.46 seconds
Started worker with PID 31620
App started successful. Use the app with http://192.168.66.162:8186/ or 192.168.66.162:8186
[Fooocus] GPU memory: max_reserved=2.080GB, max_allocated=1.973GB, reserved=2.080GB, free=12.655GB, free_torch=0.107GB, free_total=12.763GB, gpu_total=15.992GB, torch_total=2.080GB
[Comfyd] Disabling smart memory management
[Comfyd] Device: cuda:0 NVIDIA GeForce RTX 4070 Ti SUPER : cudaMallocAsync
[Topbar] No accept-language in request.headers:{'host': '192.168.66.162:8186', 'connection': 'Upgrade', 'pragma': 'no-cache', 'cache-control': 'no-cache', 'user-agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/130.0.0.0 Safari/537.36 Edg/130.0.0.0', 'upgrade': 'websocket', 'origin': 'http://192.168.66.162:8186', 'sec-websocket-version': '13', 'accept-encoding': 'gzip, deflate', 'accept-language': 'en-US,en;q=0.9,id;q=0.8', 'cookie': '_ga=GA1.1.1843782630.1725085365; _ga_R1FN4KJKJH=GS1.1.1725185769.2.0.1725185769.0.0.0', 'sec-websocket-key': 'JiuMHd82h3NrouyXOnTxOQ==', 'sec-websocket-extensions': 'permessage-deflate; client_max_window_bits'}
[Gallery] Refresh_output_catalog: A total of 257 images and 22 pages, displaying the latest 22 pages.
[Comfyd] [Prompt Server] web root: B:\tools\StabilityMatrix-win-x64\Data\Packages\SimpleSDXL\comfy\web
[Comfyd] Prestartup times for custom nodes:
[Comfyd]       0.0 seconds: B:\tools\StabilityMatrix-win-x64\Data\Packages\SimpleSDXL\comfy\custom_nodes\rgthree-comfy
[Comfyd]       0.0 seconds: B:\tools\StabilityMatrix-win-x64\Data\Packages\SimpleSDXL\comfy\custom_nodes\ComfyUI-Easy-Use
[ComfyUI-Easy-Use] server: v1.2.2 Loaded
[ComfyUI-Easy-Use] web root: B:\tools\StabilityMatrix-win-x64\Data\Packages\SimpleSDXL\comfy\custom_nodes\ComfyUI-Easy-Use\web_version/v2 Loaded
[Comfyd] Traceback (most recent call last):
  File "B:\tools\StabilityMatrix-win-x64\Data\Packages\SimpleSDXL\comfy\nodes.py", line 1994, in load_custom_node
    module_spec.loader.exec_module(module)
  File "<frozen importlib._bootstrap_external>", line 883, in exec_module
  File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
  File "B:\tools\StabilityMatrix-win-x64\Data\Packages\SimpleSDXL\comfy\custom_nodes\ComfyUI-GGUF\__init__.py", line 7, in <module>
    from .nodes import NODE_CLASS_MAPPINGS
  File "B:\tools\StabilityMatrix-win-x64\Data\Packages\SimpleSDXL\comfy\custom_nodes\ComfyUI-GGUF\nodes.py", line 3, in <module>
    import gguf
ModuleNotFoundError: No module named 'gguf'

[Comfyd] Cannot import B:\tools\StabilityMatrix-win-x64\Data\Packages\SimpleSDXL\comfy\custom_nodes\ComfyUI-GGUF module for custom nodes: No module named 'gguf'
[Comfyd] Disabling smart memory management
[Comfyd] Device: cuda:0 NVIDIA GeForce RTX 4070 Ti SUPER : cudaMallocAsync
no module 'xformers'. Processing without...
no module 'xformers'. Processing without...
[comfyui_controlnet_aux] | INFO -> Using ckpts path: B:\tools\StabilityMatrix-win-x64\Data\Packages\SimpleSDXL\models\checkpoints
[comfyui_controlnet_aux] | INFO -> Using symlinks: False
[comfyui_controlnet_aux] | INFO -> Using ort providers: ['CUDAExecutionProvider', 'DirectMLExecutionProvider', 'OpenVINOExecutionProvider', 'ROCMExecutionProvider', 'CPUExecutionProvider', 'CoreMLExecutionProvider']
B:\tools\StabilityMatrix-win-x64\Data\Packages\SimpleSDXL\comfy\custom_nodes\comfyui_controlnet_aux\node_wrappers\dwpose.py:26: UserWarning: DWPose: Onnxruntime not found or doesn't come with acceleration providers, switch to OpenCV with CPU device. DWPose might run very slowly
  warnings.warn("DWPose: Onnxruntime not found or doesn't come with acceleration providers, switch to OpenCV with CPU device. DWPose might run very slowly")
Please 'pip install xformers'
Nvidia APEX normalization not installed, using PyTorch LayerNorm

[rgthree] Loaded 42 epic nodes.
[rgthree] NOTE: Will NOT use rgthree's optimized recursive execution as ComfyUI has changed.

[Comfyd] Import times for custom nodes:
[Comfyd]    0.0 seconds: B:\tools\StabilityMatrix-win-x64\Data\Packages\SimpleSDXL\comfy\custom_nodes\websocket_image_save.py
[Comfyd]    0.0 seconds: B:\tools\StabilityMatrix-win-x64\Data\Packages\SimpleSDXL\comfy\custom_nodes\ComfyUI_bitsandbytes_NF4
[Comfyd]    0.0 seconds: B:\tools\StabilityMatrix-win-x64\Data\Packages\SimpleSDXL\comfy\custom_nodes\AIGODLIKE-COMFYUI-TRANSLATION
[Comfyd]    0.0 seconds: B:\tools\StabilityMatrix-win-x64\Data\Packages\SimpleSDXL\comfy\custom_nodes\ComfyUI-FluxExt-MZ
[Comfyd]    0.0 seconds: B:\tools\StabilityMatrix-win-x64\Data\Packages\SimpleSDXL\comfy\custom_nodes\simpleai-seamless-tiled
[Comfyd]    0.0 seconds: B:\tools\StabilityMatrix-win-x64\Data\Packages\SimpleSDXL\comfy\custom_nodes\ComfyUI_Noise
[Comfyd]    0.0 seconds: B:\tools\StabilityMatrix-win-x64\Data\Packages\SimpleSDXL\comfy\custom_nodes\ComfyUI_IPAdapter_plus
[Comfyd]    0.0 seconds (IMPORT FAILED): B:\tools\StabilityMatrix-win-x64\Data\Packages\SimpleSDXL\comfy\custom_nodes\ComfyUI-GGUF
[Comfyd]    0.0 seconds: B:\tools\StabilityMatrix-win-x64\Data\Packages\SimpleSDXL\comfy\custom_nodes\ComfyUI-layerdiffuse
[Comfyd]    0.0 seconds: B:\tools\StabilityMatrix-win-x64\Data\Packages\SimpleSDXL\comfy\custom_nodes\ComfyUI_UltimateSDUpscale
[Comfyd]    0.0 seconds: B:\tools\StabilityMatrix-win-x64\Data\Packages\SimpleSDXL\comfy\custom_nodes\ComfyUI-Advanced-ControlNet
[Comfyd]    0.0 seconds: B:\tools\StabilityMatrix-win-x64\Data\Packages\SimpleSDXL\comfy\custom_nodes\ComfyUI_essentials
[Comfyd]    0.0 seconds: B:\tools\StabilityMatrix-win-x64\Data\Packages\SimpleSDXL\comfy\custom_nodes\ComfyUI-KJNodes
[Comfyd]    0.0 seconds: B:\tools\StabilityMatrix-win-x64\Data\Packages\SimpleSDXL\comfy\custom_nodes\x-flux-comfyui
[Comfyd]    0.0 seconds: B:\tools\StabilityMatrix-win-x64\Data\Packages\SimpleSDXL\comfy\custom_nodes\ComfyUI-IC-Light
[Comfyd]    0.0 seconds: B:\tools\StabilityMatrix-win-x64\Data\Packages\SimpleSDXL\comfy\custom_nodes\ComfyUI-Kolors-MZ
[Comfyd]    0.0 seconds: B:\tools\StabilityMatrix-win-x64\Data\Packages\SimpleSDXL\comfy\custom_nodes\ComfyUI_ExtraModels
[Comfyd]    0.0 seconds: B:\tools\StabilityMatrix-win-x64\Data\Packages\SimpleSDXL\comfy\custom_nodes\rgthree-comfy
[Comfyd]    0.1 seconds: B:\tools\StabilityMatrix-win-x64\Data\Packages\SimpleSDXL\comfy\custom_nodes\ComfyUI-KwaiKolorsWrapper
[Comfyd]    0.1 seconds: B:\tools\StabilityMatrix-win-x64\Data\Packages\SimpleSDXL\comfy\custom_nodes\PuLID_ComfyUI
[Comfyd]    0.1 seconds: B:\tools\StabilityMatrix-win-x64\Data\Packages\SimpleSDXL\comfy\custom_nodes\comfyui_controlnet_aux
[Comfyd]    0.2 seconds: B:\tools\StabilityMatrix-win-x64\Data\Packages\SimpleSDXL\comfy\custom_nodes\ComfyUI-Easy-Use
[Comfyd]    0.3 seconds: B:\tools\StabilityMatrix-win-x64\Data\Packages\SimpleSDXL\comfy\custom_nodes\ComfyUI-BrushNet
[Comfyd]    0.8 seconds: B:\tools\StabilityMatrix-win-x64\Data\Packages\SimpleSDXL\comfy\custom_nodes\ComfyUI_InstantID
[Comfyd]    1.6 seconds: B:\tools\StabilityMatrix-win-x64\Data\Packages\SimpleSDXL\comfy\custom_nodes\ComfyUI-SUPIR
[Comfyd] 
[Comfyd] Starting Comfyd server!

adding something rare to the prompt
(photography art stylized by Kati Horna:0.8) , landscape of a Neon Rooftop Market and Fields of heather, it is Messy, dynamic, Fujicolor Superia X-TRA 400, F/1.8
[Fooocus] GPU memory: max_reserved=2.080GB, max_allocated=1.973GB, reserved=2.080GB, free=12.655GB, free_torch=0.107GB, free_total=12.763GB, gpu_total=15.992GB, torch_total=2.080GB
[TaskEngine] Task_class:Fooocus, Task_name:default, Task_method:text2image
[TaskEngine] Enable Fooocus backend.
[Comfyd] Comfyd freeing!
[Parameters] Adaptive CFG = 7
[Parameters] CLIP Skip = 2
[Parameters] Sharpness = 2
[Parameters] ControlNet Softness = 0.25
[Parameters] ADM Scale = 1.5 : 0.8 : 0.3
[Parameters] Seed = 4364341327904623083
[Parameters] CFG = 4
[Fooocus] Loading control models ...
[Parameters] Sampler = dpmpp_2m_sde_gpu - karras
[Parameters] Steps = 30 - 15
[Fooocus] Initializing ...
[Fooocus] Loading models ...
Refiner unloaded.
[Fooocus] Processing prompts ...
[Wildcards] Copmile text in prompt to arrays: (photography art stylized by Kati Horna:0.8), landscape of a Neon Rooftop Market and Fields of heather, it is Messy, dynamic, Fujicolor Superia X-TRA 400, F/1.8 -> arrays:[], mult:0
[Fooocus] Preparing Fooocus text #1 ...
B:\tools\StabilityMatrix-win-x64\Data\Packages\SimpleSDXL\venv\lib\site-packages\transformers\models\gpt2\modeling_gpt2.py:650: UserWarning: 1Torch was not compiled with flash attention. (Triggered internally at ..\aten\src\ATen\native\transformers\cuda\sdp_utils.cpp:455.)
  attn_output = torch.nn.functional.scaled_dot_product_attention(
[Prompt Expansion] (photography art stylized by Kati Horna:0.8), landscape of a Neon Rooftop Market and Fields of heather, it is Messy, dynamic, Fujicolor Superia X-TRA 400, F/1.8, light reflected sharp focus, beautiful detailed, very inspirational, intricate, innocent, inspiring, symmetry, fine
[Fooocus] Preparing Fooocus text #2 ...
[Prompt Expansion] (photography art stylized by Kati Horna:0.8), landscape of a Neon Rooftop Market and Fields of heather, it is Messy, dynamic, Fujicolor Superia X-TRA 400, F/1.8, light, reflected moving, vibrant colors, extremely detailed, beautiful, intricate, very inspirational, inspiring
[Fooocus] Encoding positive #1 ...
[Fooocus Model Management] Moving model(s) has taken 0.13 seconds
[Fooocus] Encoding positive #2 ...
[Fooocus] Encoding negative #1 ...
[Fooocus] Encoding negative #2 ...
[Parameters] Denoising Strength = 1.0
[Parameters] Initial Latent shape: Image Space (896, 1152)
Preparation time: 2.02 seconds
Using karras scheduler.
[Fooocus] GPU memory: max_reserved=2.105GB, max_allocated=1.989GB, reserved=1.922GB, free=12.767GB, free_torch=0.190GB, free_total=12.956GB, gpu_total=15.992GB, torch_total=1.922GB
[Fooocus] Preparing Fooocus task 1/2 ...
[Sampler] refiner_swap_method = joint
[Sampler] sigma_min = 0.0291671771556139, sigma_max = 14.614643096923828
Requested to load SDXL
Loading 1 new model
[Fooocus Model Management] Moving model(s) has taken 1.94 seconds
100%|██████████| 30/30 [00:07<00:00,  4.07it/s]
Requested to load AutoencoderKL
Loading 1 new model
[Fooocus Model Management] Moving model(s) has taken 0.15 seconds
[Fooocus] Saving image 1/2 to system ...
[ModelInfo] Calculate hash for B:\tools\StabilityMatrix-win-x64\Data\Models\StableDiffusion\juggernautXL_v9Rundiffusionphoto2.safetensors
[ModelInfo] Calculate addnet hash for B:\tools\StabilityMatrix-win-x64\Data\Models\StableDiffusion\juggernautXL_v9Rundiffusionphoto2.safetensors
Image generated with private log at: B:\tools\StabilityMatrix-win-x64\Data\Packages\SimpleSDXL\outputs\2024-11-12\log.html
Generating and saving time: 20.16 seconds
[Fooocus] GPU memory: max_reserved=5.496GB, max_allocated=5.119GB, reserved=3.262GB, free=11.407GB, free_torch=3.082GB, free_total=14.489GB, gpu_total=15.992GB, torch_total=3.262GB
[Fooocus] Preparing Fooocus task 2/2 ...
[Sampler] refiner_swap_method = joint
[Sampler] sigma_min = 0.0291671771556139, sigma_max = 14.614643096923828
Requested to load SDXL
Loading 1 new model
[Fooocus Model Management] Moving model(s) has taken 1.17 seconds
100%|██████████| 30/30 [00:07<00:00,  4.28it/s]
Requested to load AutoencoderKL
Loading 1 new model
[Fooocus Model Management] Moving model(s) has taken 0.13 seconds
[Fooocus] Saving image 2/2 to system ...
Image generated with private log at: B:\tools\StabilityMatrix-win-x64\Data\Packages\SimpleSDXL\outputs\2024-11-12\log.html
Generating and saving time: 8.61 seconds
[Fooocus] GPU memory: max_reserved=5.506GB, max_allocated=5.131GB, reserved=3.291GB, free=11.378GB, free_torch=3.100GB, free_total=14.478GB, gpu_total=15.992GB, torch_total=3.291GB
[Enhance] Skipping, preconditions aren't met
Processing time (total): 28.77 seconds
Requested to load SDXLClipModel
Requested to load GPT2LMHeadModel
Loading 2 new models
Total time: 30.87 seconds
[Gallery] Refresh_output_catalog: A total of 259 images and 22 pages, displaying the latest 22 pages.
[Gallery] Parse_html_log: loaded 2 image_infos of 24-11-12.
[Gallery] Refresh_images_catalog: loaded 2 image_items of 24-11-12.
[Gallery] Parse_html_log: loaded 2 image_infos of 24-11-12.
[Fooocus Model Management] Moving model(s) has taken 0.38 seconds

Additional information

i installed simplesdxl with stabilitymatrix

DavidDragonsage commented 2 days ago

Are you sure you are using the Flux preset, selected from one of the buttons above the main canvas? Its getting late here and I know I am not thinking clearly, but I could not find where a Flux model was loaded in the console log. Which Flux base model are you trying to use?

If are stuck, I would try installing SimpleSDXL2 normally rather than going through StabilityMatrix. It could be that Stability has not kept up with the latest version of SimpleSDXL2. I did have StabilityMatrix problems when I tried installing a different generative UI platform.

If you are having trouble with the installation, please refer to our installation guide at the Facebook Pure Fooocus group: https://www.facebook.com/groups/fooocus/learning_content/?filter=1078416300308780&post=1031418935004333 Facebook links are not always reliable. If the link does not work, go to Guides section and scroll down to Guide 11. And of course we will help as much as we can.

orangpelupa commented 2 days ago

Are you sure you are using the Flux preset, selected from one of the buttons above the main canvas? Its getting late here and I know I am not thinking clearly, but I could not find where a Flux model was loaded in the console log. Which Flux base model are you trying to use?

If are stuck, I would try installing SimpleSDXL2 normally rather than going through StabilityMatrix. It could be that Stability has not kept up with the latest version of SimpleSDXL2. I did have StabilityMatrix problems when I tried installing a different generative UI platform.

If you are having trouble with the installation, please refer to our installation guide at the Facebook Pure Fooocus group: https://www.facebook.com/groups/fooocus/learning_content/?filter=1078416300308780&post=1031418935004333 Facebook links are not always reliable. If the link does not work, go to Guides section and scroll down to Guide 11. And of course we will help as much as we can.

i tried simplesdxl2 from huggingface, and initially it installs. but after i interrupted the installation when it was downloading juggernaut model, running the install bat give me this file not found error

the same error even after i extracted simplesdxl2 on differrent folder.

DavidDragonsage commented 2 days ago

Yes, it is annoying that it downloads that model, but it is probably better to put up with it and clean up later. If you are like me in that respect, you probably already have that model somewhere else.

Like mainline Fooocus, Simple wants you to have something to try out when you run the UI for the first time - well in fact either of them will complain if you do not have the default model somewhere whenever you startup the UI. But you can edit the paths in config.txt later if you want Juggernaut in a different location. At least Simple uses Juggernaut XI which is a very good model - not the best but in the top ten.

At this point I would remove Simple completely and start again.

metercai / SimpleSDXL