[Bug]: 启动可图模型，llms_model = quant4 参数不能改吗？

Checklist

[ ] The issue has not been resolved by following the troubleshooting guide
[ ] The issue exists on a clean installation of Fooocus
[ ] The issue exists in the current version of Fooocus
[ ] The issue has not been reported before recently
[ ] The issue has been reported before but has not been fixed yet

What happened?

之前在 ComfyUI 测试过，我显卡只能用 fp16 方式加载 chatglm3 模型，4bit/8bit 都会报错，我已经修改了： SimpleSDXL\workflows 目录下的 kolors_text2image1_api.json 和 kolors_text2image1_flow.json 里面的 widgets_values": ["fp16"]，可以，重新启动，命令行窗口显示的参数依然是： [ComfyClient] Ready ComfyTask to process: workflow=kolors_text2image1 prompt = 一个女孩 negative_prompt = width = 1024 height = 1024 base_model = kolors_unet_fp16.safetensors scheduler = EulerDiscreteScheduler cfg_scale = 5 steps = 25 denoise = 1.0 seed = 4604319255044166361 llms_model = quant4

Steps to reproduce the problem

选择可图模型，2. 输入提示词， 3. 运行， 4. 报错

What should have happened?

chatglm3 这个模型在不同环境加载有很多问题，最后有个量化参数的配置选项。

What browsers do you use to access Fooocus?

Google Chrome

Where are you running Fooocus?

Locally

What operating system are you using?

windows 11 x64，24GB GPU

Console logs

E:\SimpleSDXL2>run_SImpleSDXL_dev.bat
SimpleSDXL_dev: Already up-to-date, 50ff1ea
[System ARGV] ['SimpleSDXL\\entry_with_update.py', '--dev']
Python 3.10.11 (tags/v3.10.11:7d4cc5a, Apr  5 2023, 00:38:17) [MSC v.1929 64 bit (AMD64)]
Fooocus version: 2.5.0
Comfy version: 1281f93
SimpleSDXL_dev version: v20240630.50ff1ea
Uninstall simpleai_base 0.3.16
Install simpleai_base 0.3.16
Checking ...
[SimpleAI] GPU: NVIDIA Quadro P3200 with Max-Q Design
NVIDIA Tesla P40, RAM: 65317MB, SWAP: 0MB, VRAM: 6144MB, DiskFree: 587899MB
[SimpleAI] local_did/本地身份ID: gPYPmKXZCGmgxGrUCEiquzBovT6
Load config data from E:\SimpleSDXL2\SimpleSDXL\config.txt.
Loaded preset: E:\SimpleSDXL2\SimpleSDXL\presets\default.json
Total VRAM 24474 MB, total RAM 65318 MB
xformers version: 0.0.26
Set vram state to: NORMAL_VRAM
Always offload VRAM
Device: cuda:0 Tesla P40 : native
VAE dtype: torch.float32
[Cleanup] Attempting to delete content of temp dir C:\Users\ADMINI~1\AppData\Local\Temp\fooocus
[Cleanup] Cleanup successful
Using xformers cross attention
Refiner unloaded.
model_type EPS
UNet ADM Dimension 2816
[Wildcards] Refresh and Load 27/27 wildcards: animal, bodytypes, buildings, chairs, color, containers, coverings, flora, flower, focus, foods, hairstyles, jobs, materials, minilocations, moods, nationality, objects, objectstohold, outfits, patterns, poses, rpg, shotsizes, space, timeperiods, vehicles.
[OneButtonPrompt] The customized module:OneButtonPrompt is initializing ...
Running on local URL:  http://192.168.1.8:8186

To create a public link, set `share=True` in `launch()`.
IMPORTANT: You are using gradio version 3.41.2, however version 4.29.0 is available, please upgrade.
--------
[Gallery] Refresh_output_catalog: loaded 1 images_catalogs.
[Gallery] Refresh_output_catalog: loaded 1 images_catalogs.
[Gallery] Refresh_output_catalog: loaded 1 images_catalogs.
Using xformers attention in VAE
Working with z of shape (1, 4, 32, 32) = 4096 dimensions.
Using xformers attention in VAE
[Topbar] Reset_context: preset=default-->Kolors, theme=dark, lang=cn
[Topbar] Ready to reset generation params session based by preset:Kolors.
extra {'cond_stage_model.clip_l.logit_scale', 'cond_stage_model.clip_l.text_projection'}
left over keys: dict_keys(['cond_stage_model.clip_l.transformer.text_model.embeddings.position_ids'])
Base model loaded: E:\SimpleSDXL2\SimpleSDXL\models\checkpoints\juggernaut-X-RunDiffusion-NSFW.safetensors
VAE loaded: None
Request to load LoRAs [('sd_xl_offset_example-lora_1.0.safetensors', 0.1)] for model [E:\SimpleSDXL2\SimpleSDXL\models\checkpoints\juggernaut-X-RunDiffusion-NSFW.safetensors].
Loaded LoRA [E:\SimpleSDXL2\SimpleSDXL\models\loras\sd_xl_offset_example-lora_1.0.safetensors] for UNet [E:\SimpleSDXL2\SimpleSDXL\models\checkpoints\juggernaut-X-RunDiffusion-NSFW.safetensors] with 788 keys at weight 0.1.
Fooocus V2 Expansion: Vocab with 642 words.
Fooocus Expansion engine loaded for cpu, use_fp16 = False.
Requested to load SDXLClipModel
Requested to load GPT2LMHeadModel
Loading 2 new models
Started worker with PID 21760
App started successful. Use the app with http://192.168.1.8:8186/ or 192.168.1.8:8186
Please 'pip install apex'
[Comfyd] Import times for custom nodes:
[Comfyd]    0.0 seconds: E:\SimpleSDXL2\SimpleSDXL\comfy\custom_nodes\websocket_image_save.py
[Comfyd]    0.0 seconds: E:\SimpleSDXL2\SimpleSDXL\comfy\custom_nodes\AIGODLIKE-COMFYUI-TRANSLATION
[Comfyd]    0.0 seconds: E:\SimpleSDXL2\SimpleSDXL\comfy\custom_nodes\ComfyUI-Kolors-MZ
[Comfyd]    0.0 seconds: E:\SimpleSDXL2\SimpleSDXL\comfy\custom_nodes\ComfyUI_IPAdapter_plus
[Comfyd]    0.0 seconds: E:\SimpleSDXL2\SimpleSDXL\comfy\custom_nodes\ComfyUI-layerdiffuse
[Comfyd]    0.0 seconds: E:\SimpleSDXL2\SimpleSDXL\comfy\custom_nodes\ComfyUI_UltimateSDUpscale
[Comfyd]    0.0 seconds: E:\SimpleSDXL2\SimpleSDXL\comfy\custom_nodes\ComfyUI-IC-Light
[Comfyd]    0.0 seconds: E:\SimpleSDXL2\SimpleSDXL\comfy\custom_nodes\ComfyUI-Advanced-ControlNet
[Comfyd]    0.1 seconds: E:\SimpleSDXL2\SimpleSDXL\comfy\custom_nodes\ComfyUI-KwaiKolorsWrapper
[Comfyd]    0.1 seconds: E:\SimpleSDXL2\SimpleSDXL\comfy\custom_nodes\ComfyUI_essentials
[Comfyd]    0.2 seconds: E:\SimpleSDXL2\SimpleSDXL\comfy\custom_nodes\PuLID_ComfyUI
[Comfyd]    0.3 seconds: E:\SimpleSDXL2\SimpleSDXL\comfy\custom_nodes\ComfyUI-Easy-Use
[Comfyd]    0.3 seconds: E:\SimpleSDXL2\SimpleSDXL\comfy\custom_nodes\ComfyUI-BrushNet
[Comfyd]    0.7 seconds: E:\SimpleSDXL2\SimpleSDXL\comfy\custom_nodes\ComfyUI-KJNodes
[Comfyd]    1.8 seconds: E:\SimpleSDXL2\SimpleSDXL\comfy\custom_nodes\comfyui-hydit
[Comfyd]    2.4 seconds: E:\SimpleSDXL2\SimpleSDXL\comfy\custom_nodes\ComfyUI-SUPIR
[Comfyd]    9.8 seconds: E:\SimpleSDXL2\SimpleSDXL\comfy\custom_nodes\ComfyUI_InstantID
[Comfyd]
[Comfyd] Starting Comfyd server!

[TaskEngine] task_class:Kolors, task_name:Kolors, task_method:kolors_text2image1
[Comfyd] Comfyd is running!
Refiner disabled because base model and refiner are same.
[Parameters] Adaptive CFG = 7
[Parameters] CLIP Skip = 2
[Parameters] Sharpness = 2
[Parameters] ControlNet Softness = 0.25
[Parameters] ADM Scale = 1.5 : 0.8 : 0.3
[Parameters] Seed = 4604319255044166361
[Parameters] CFG = 5
[Fooocus] Loading control models ...
[Parameters] Sampler =  - EulerDiscreteScheduler
[Parameters] Steps = 25 - 15
[Fooocus] Initializing ...
[Fooocus] Processing prompts ...
[Wildcards] Copmile text in prompt to arrays: 一个女孩 -> arrays:[], mult:0
[Parameters] Denoising Strength = 1.0
[Parameters] Initial Latent shape: Image Space (1024, 1024)
Preparation time: 0.01 seconds
Using EulerDiscreteScheduler scheduler.
[Fooocus] GPU Memory, max: 0.000GB, allocated:0.000MB, chached: 0.000GB
[Fooocus] Preparing Kolors task 1/1 ...
[ComfyClient] Ready ComfyTask to process: workflow=kolors_text2image1
    prompt = 一个女孩
    negative_prompt =
    width = 1024
    height = 1024
    base_model = kolors_unet_fp16.safetensors
    scheduler = EulerDiscreteScheduler
    cfg_scale = 5
    steps = 25
    denoise = 1.0
    seed = 4604319255044166361
    llms_model = quant4
[Comfyd] got prompt
[ComfyClient] Request and get ComfyTask_id:d686d864-d390-42c3-8edb-230b2f239f5f
Load UNET...
Kolors-unet: RAM allocated = 10.295GB
Load TEXT_ENCODER..., quant4, cpu
Loading checkpoint shards: 100%|███████████████████████████████████████████████████████| 7/7 [00:21<00:00,  3.03s/it]
[Comfyd] !!! Exception during processing!!! CUDA error: operation not supported
CUDA kernel errors might be asynchronously reported at some other API call, so the stacktrace below might be incorrect.
For debugging consider passing CUDA_LAUNCH_BLOCKING=1.
Compile with `TORCH_USE_CUDA_DSA` to enable device-side assertions.

[Comfyd] Traceback (most recent call last):
  File "E:\SimpleSDXL2\SimpleSDXL\comfy\execution.py", line 151, in recursive_execute
    output_data, output_ui = get_output_data(obj, input_data_all)
  File "E:\SimpleSDXL2\SimpleSDXL\comfy\execution.py", line 81, in get_output_data
    return_values = map_node_over_list(obj, input_data_all, obj.FUNCTION, allow_interrupt=True)
  File "E:\SimpleSDXL2\SimpleSDXL\comfy\execution.py", line 74, in map_node_over_list
    results.append(getattr(obj, func)(**slice_dict(input_data_all, i)))
  File "E:\SimpleSDXL2\SimpleSDXL\comfy\custom_nodes\ComfyUI-KwaiKolorsWrapper\nodes.py", line 194, in loadmodel
    text_encoder.quantize(4)
  File "E:\SimpleSDXL2\SimpleSDXL\comfy\custom_nodes\ComfyUI-KwaiKolorsWrapper\kolors\models\modeling_chatglm.py", line 852, in quantize
    quantize(self.encoder, weight_bit_width)
  File "E:\SimpleSDXL2\SimpleSDXL\comfy\custom_nodes\ComfyUI-KwaiKolorsWrapper\kolors\models\quantization.py", line 157, in quantize
    weight=layer.self_attention.query_key_value.weight.to(torch.cuda.current_device()),
RuntimeError: CUDA error: operation not supported
CUDA kernel errors might be asynchronously reported at some other API call, so the stacktrace below might be incorrect.
For debugging consider passing CUDA_LAUNCH_BLOCKING=1.
Compile with `TORCH_USE_CUDA_DSA` to enable device-side assertions.

[Comfyd] Max_memory_allocated: 0.00GB
[Comfyd] Prompt executed in 27.53 seconds
[ComfyClient] The ComfyTask:d686d864-d390-42c3-8edb-230b2f239f5f has finished: 0
[Fooocus] Saving image 1/1 to system ...
Generating and saving time: 27.75 seconds
[Fooocus] GPU Memory, max: 0.000GB, allocated:0.000MB, chached: 0.000GB
[Enhance] Skipping, preconditions aren't met
Processing time (total): 27.75 seconds
[Comfyd] Comfyd keeps active!
Total time: 27.99 seconds
[Gallery] Refresh_output_catalog: loaded 1 images_catalogs.
[Gallery] Parse_html_log: loaded 1 image_infos of 24-07-21.
[Gallery] Refresh_images_catalog: loaded 1 image_items of 24-07-21.
[Gallery] Parse_html_log: loaded 1 image_infos of 24-07-21.
Keyboard interruption in main thread... closing server.
[Comfyd]

Additional information

之前在ComfyUI中，试过 fp16 加载没问题。这次用大佬B站推荐的一键包，从hf-mirror.com 拉下来的所有压缩包解压后运行，apex 本地编译安装后，还是会报那个 pip install apex 提示。

metercai / SimpleSDXL