Open hok0901 opened 2 months ago
I ran into this issue as well, so i started testing several commits while doing git bisect. It seemed to indicate that the issue started at commit 9bc2d04ca942d7d71d1f4c4c2e94945ecdf0e4a5:
9bc2d04ca942d7d71d1f4c4c2e94945ecdf0e4a5 is the first bad commit
commit 9bc2d04ca942d7d71d1f4c4c2e94945ecdf0e4a5
Merge: d7151b4d 9f5a27ca
Author: Panchovix <francisco.jaqueat@gmail.com>
Date: Mon Aug 19 16:58:52 2024 -0400
Merge pull request #1310 from lllyasviel/more_schedulers
Add Align Your Steps GITS, AYS 11 Steps and AYS 32 Steps Schedulers.
modules/sd_schedulers.py | 76 ++++++++++++++++++++++++++++++++++++++++++++++++
1 file changed, 76 insertions(+)
for me, bb58520a4c5bf268b567a6f7cb58901bd942ae59 is the first bad one, before I get the 'NoneType' object is not iterable
same problem here
you shoud enabel Settings->Stable Diffusion3->Enable T5
Is SD3 currently working for you in Forge? If so, maybe you share which Forge version number you use.
'Enable T5' - no matter whether enabled or not (!)* - delivers in my setup the same error message at the end as OP's, the "t5.py" script is the one that throws an error in the "activations" part (the last lines), everything seems to be fine until...
self.DenseReluDense = T5DenseGatedActDense(model_dim, ff_dim, ff_activation)
File "D:\sd-forge\backend\nn\t5.py", line 44, in init self.act = activations[ff_activation] KeyError: 'gelu_new' 'gelu_new'
I'm on version: f2.0.1v1.10.1-previous-528-ge55cde9b (three days old, Python 3.10.11, torch: 2.3.1+cu121), had the same problem with the version before, but don't recall exactly when I first noticed it.
What happens after line 44 in "t.5.py"? I don't have a clue but the script has another 168 lines. The script continues next with...
def forward(self, x): hidden_gelu = self.act(self.wi_0(x)) hidden_linear = self.wi_1(x) x = hidden_gelu * hidden_linear x = self.wo(x) return x
*= And if T5 is enabled in the SD3 setting it first delivers a series of error messages:
ERROR: Exception in ASGI application Traceback (most recent call last): File "D:\sd-forge\venv\lib\site-packages\starlette\middleware\exceptions.py", line 68, in call await self.app(scope, receive, sender) File "D:\sd-forge\venv\lib\site-packages\fastapi\middleware\asyncexitstack.py", line 20, in call raise e File "D:\sd-forge\venv\lib\site-packages\fastapi\middleware\asyncexitstack.py", line 17, in call await self.app(scope, receive, send) File "D:\sd-forge\venv\lib\site-packages\starlette\routing.py", line 718, in call await route.handle(scope, receive, send) File "D:\sd-forge\venv\lib\site-packages\starlette\routing.py", line 276, in handle await self.app(scope, receive, send) File "D:\sd-forge\venv\lib\site-packages\starlette\routing.py", line 69, in app await response(scope, receive, send) File "D:\sd-forge\venv\lib\site-packages\starlette\responses.py", line 270, in call async with anyio.create_task_group() as task_group: File "D:\sd-forge\venv\lib\site-packages\anyio_backends_asyncio.py", line 597, in aexit raise exceptions[0] File "D:\sd-forge\venv\lib\site-packages\starlette\responses.py", line 273, in wrap await func() File "D:\sd-forge\venv\lib\site-packages\starlette\responses.py", line 262, in stream_response async for chunk in self.body_iterator: File "D:\sd-forge\venv\lib\site-packages\gradio\routes.py", line 956, in sse_stream raise e File "D:\sd-forge\venv\lib\site-packages\gradio\routes.py", line 891, in sse_stream raise HTTPException( fastapi.exceptions.HTTPException
The above exception was the direct cause of the following exception:
Traceback (most recent call last): File "D:\sd-forge\venv\lib\site-packages\uvicorn\protocols\http\h11_impl.py", line 406, in run_asgi result = await app( # type: ignore[func-returns-value] File "D:\sd-forge\venv\lib\site-packages\uvicorn\middleware\proxy_headers.py", line 70, in call return await self.app(scope, receive, send) File "D:\sd-forge\venv\lib\site-packages\fastapi\applications.py", line 1106, in call await super().call(scope, receive, send) File "D:\sd-forge\venv\lib\site-packages\starlette\applications.py", line 122, in call await self.middleware_stack(scope, receive, send) File "D:\sd-forge\venv\lib\site-packages\starlette\middleware\errors.py", line 184, in call raise exc File "D:\sd-forge\venv\lib\site-packages\starlette\middleware\errors.py", line 162, in call await self.app(scope, receive, _send) File "D:\sd-forge\venv\lib\site-packages\gradio\route_utils.py", line 724, in call await self.app(scope, receive, send) File "D:\sd-forge\venv\lib\site-packages\starlette\middleware\exceptions.py", line 83, in call raise RuntimeError(msg) from exc RuntimeError: Caught handled exception, but response already started.
I'm getting this error as well.
StateDict Keys: {'transformer': 780, 'vae': 244, 'text_encoder': 196, 'text_encoder_2': 220, 'ignore': 0}
Using Default T5 Data Type: torch.float16
Traceback (most recent call last):
File "/var/home/$USER/stable-diffusion-webui-forge/modules_forge/main_thread.py", line 30, in work
self.result = self.func(*self.args, **self.kwargs)
File "/var/home/$USER/stable-diffusion-webui-forge/extensions/sd-webui-fsr-integration/modules/txt2img.py", line 123, in txt2img_function
processed = processing.process_images(p)
File "/var/home/$USER/stable-diffusion-webui-forge/modules/processing.py", line 800, in process_images
p.sd_model, just_reloaded = forge_model_reload()
File "/var/home/$USER/stable-diffusion-webui-forge/venv/lib64/python3.10/site-packages/torch/utils/_contextlib.py", line 116, in decorate_context
return func(*args, **kwargs)
File "/var/home/$USER/stable-diffusion-webui-forge/modules/sd_models.py", line 494, in forge_model_reload
sd_model = forge_loader(state_dict, additional_state_dicts=additional_state_dicts)
File "/var/home/$USER/stable-diffusion-webui-forge/venv/lib64/python3.10/site-packages/torch/utils/_contextlib.py", line 116, in decorate_context
return func(*args, **kwargs)
File "/var/home/$USER/stable-diffusion-webui-forge/backend/loader.py", line 283, in forge_loader
component = load_huggingface_component(estimated_config, component_name, lib_name, cls_name, local_path, component_sd)
File "/var/home/$USER/stable-diffusion-webui-forge/backend/loader.py", line 103, in load_huggingface_component
model = IntegratedT5(config)
File "/var/home/$USER/stable-diffusion-webui-forge/backend/nn/t5.py", line 211, in __init__
self.transformer = T5(config)
File "/var/home/$USER/stable-diffusion-webui-forge/backend/nn/t5.py", line 199, in __init__
self.encoder = T5Stack(self.num_layers, model_dim, model_dim, config["d_ff"], config["dense_act_fn"], config["is_gated_act"], config["num_heads"], config["model_type"] != "umt5")
File "/var/home/$USER/stable-diffusion-webui-forge/backend/nn/t5.py", line 172, in __init__
[T5Block(model_dim, inner_dim, ff_dim, ff_activation, gated_act, num_heads, relative_attention_bias=((not relative_attention) or (i == 0))) for i in range(num_layers)]
File "/var/home/$USER/stable-diffusion-webui-forge/backend/nn/t5.py", line 172, in <listcomp>
[T5Block(model_dim, inner_dim, ff_dim, ff_activation, gated_act, num_heads, relative_attention_bias=((not relative_attention) or (i == 0))) for i in range(num_layers)]
File "/var/home/$USER/stable-diffusion-webui-forge/backend/nn/t5.py", line 159, in __init__
self.layer.append(T5LayerFF(model_dim, ff_dim, ff_activation, gated_act))
File "/var/home/$USER/stable-diffusion-webui-forge/backend/nn/t5.py", line 58, in __init__
self.DenseReluDense = T5DenseGatedActDense(model_dim, ff_dim, ff_activation)
File "/var/home/$USER/stable-diffusion-webui-forge/backend/nn/t5.py", line 44, in __init__
self.act = activations[ff_activation]
KeyError: 'gelu_new'
'gelu_new'
So Flux is pretty much completely broken atm using this project.
ControlNet preprocessor location: G:\webui_forge_cu121_torch231\webui\models\ControlNetPreprocessor 2024-08-16 16:49:29,387 - ControlNet - INFO - ControlNet UI callback registered. Model selected: {'checkpoint_info': {'filename': 'G:\ComfyUI\models\checkpoints\SD3\sd3_medium_incl_clips_t5xxlfp8.safetensors', 'hash': '3b2e42cf'}, 'additional_modules': [], 'unet_storage_dtype': torch.float8_e4m3fn} Running on local URL: http://127.0.0.1:7860
To create a public link, set
[T5Block(model_dim, inner_dim, ff_dim, ff_activation, gated_act, num_heads, relative_attention_bias=((not relative_attention) or (i == 0))) for i in range(num_layers)]
File "G:\webui_forge_cu121_torch231\webui\backend\nn\t5.py", line 159, in init
self.layer.append(T5LayerFF(model_dim, ff_dim, ff_activation, gated_act))
File "G:\webui_forge_cu121_torch231\webui\backend\nn\t5.py", line 58, in init
self.DenseReluDense = T5DenseGatedActDense(model_dim, ff_dim, ff_activation)
File "G:\webui_forge_cu121_torch231\webui\backend\nn\t5.py", line 44, in init
self.act = activations[ff_activation]
KeyError: 'gelu_new'
'gelu_new'
Loading Model: {'checkpoint_info': {'filename': 'G:\ComfyUI\models\checkpoints\SD3\sd3_medium_incl_clips.safetensors', 'hash': '080136ef'}, 'additional_modules': ['G:\ComfyUI\models\vae\t5xxl_fp8_e4m3fn.safetensors'], 'unet_storage_dtype': torch.float8_e4m3fn}
StateDict Keys: {'unet': 491, 'vae': 244, 'text_encoder': 196, 'text_encoder_2': 517, 'text_encoder_3': 220, 'ignore': 0}
Using Detected T5 Data Type: torch.float8_e4m3fn
Traceback (most recent call last):
File "G:\webui_forge_cu121_torch231\webui\modules_forge\main_thread.py", line 30, in work
self.result = self.func(*self.args, self.kwargs)
File "G:\webui_forge_cu121_torch231\webui\modules\txt2img.py", line 110, in txt2img_function
processed = processing.process_images(p)
File "G:\webui_forge_cu121_torch231\webui\modules\processing.py", line 789, in process_images
p.sd_model, just_reloaded = forge_model_reload()
File "C:\Program Files\Python310\lib\site-packages\torch\utils_contextlib.py", line 115, in decorate_context
return func(*args, *kwargs)
File "G:\webui_forge_cu121_torch231\webui\modules\sd_models.py", line 501, in forge_model_reload
sd_model = forge_loader(state_dict, additional_state_dicts=additional_state_dicts)
File "C:\Program Files\Python310\lib\site-packages\torch\utils_contextlib.py", line 115, in decorate_context
return func(args, kwargs)
File "G:\webui_forge_cu121_torch231\webui\backend\loader.py", line 261, in forge_loader
component = load_huggingface_component(estimated_config, component_name, lib_name, cls_name, local_path, component_sd)
File "G:\webui_forge_cu121_torch231\webui\backend\loader.py", line 94, in load_huggingface_component
model = IntegratedT5(config)
File "G:\webui_forge_cu121_torch231\webui\backend\nn\t5.py", line 211, in init
self.transformer = T5(config)
File "G:\webui_forge_cu121_torch231\webui\backend\nn\t5.py", line 199, in init
self.encoder = T5Stack(self.num_layers, model_dim, model_dim, config["d_ff"], config["dense_act_fn"], config["is_gated_act"], config["num_heads"], config["model_type"] != "umt5")
File "G:\webui_forge_cu121_torch231\webui\backend\nn\t5.py", line 172, in init
[T5Block(model_dim, inner_dim, ff_dim, ff_activation, gated_act, num_heads, relative_attention_bias=((not relative_attention) or (i == 0))) for i in range(num_layers)]
File "G:\webui_forge_cu121_torch231\webui\backend\nn\t5.py", line 172, in
[T5Block(model_dim, inner_dim, ff_dim, ff_activation, gated_act, num_heads, relative_attention_bias=((not relative_attention) or (i == 0))) for i in range(num_layers)]
File "G:\webui_forge_cu121_torch231\webui\backend\nn\t5.py", line 159, in init
self.layer.append(T5LayerFF(model_dim, ff_dim, ff_activation, gated_act))
File "G:\webui_forge_cu121_torch231\webui\backend\nn\t5.py", line 58, in init
self.DenseReluDense = T5DenseGatedActDense(model_dim, ff_dim, ff_activation)
File "G:\webui_forge_cu121_torch231\webui\backend\nn\t5.py", line 44, in init
self.act = activations[ff_activation]
KeyError: 'gelu_new'
'gelu_new'
share=True
inlaunch()
. Startup time: 21.5s (prepare environment: 6.0s, launcher: 1.8s, import torch: 4.5s, initialize shared: 0.7s, other imports: 0.9s, list SD models: 0.2s, load scripts: 1.8s, create ui: 2.4s, gradio launch: 1.0s, add APIs: 2.2s). Environment vars changed: {'stream': False, 'inference_memory': 1024.0, 'pin_shared_memory': False} Model selected: {'checkpoint_info': {'filename': 'G:\ComfyUI\models\checkpoints\SD3\sd3_medium_incl_clips_t5xxlfp8.safetensors', 'hash': '3b2e42cf'}, 'additional_modules': [], 'unet_storage_dtype': None} Model selected: {'checkpoint_info': {'filename': 'G:\ComfyUI\models\checkpoints\SD3\sd3_medium_incl_clips_t5xxlfp8.safetensors', 'hash': '3b2e42cf'}, 'additional_modules': [], 'unet_storage_dtype': torch.float8_e4m3fn} Model selected: {'checkpoint_info': {'filename': 'G:\ComfyUI\models\checkpoints\SD3\sd3_medium.safetensors', 'hash': '9b88d133'}, 'additional_modules': [], 'unet_storage_dtype': torch.float8_e4m3fn} Model selected: {'checkpoint_info': {'filename': 'G:\ComfyUI\models\checkpoints\SD3\sd3_medium_incl_clips.safetensors', 'hash': '080136ef'}, 'additional_modules': [], 'unet_storage_dtype': torch.float8_e4m3fn} Model selected: {'checkpoint_info': {'filename': 'G:\ComfyUI\models\checkpoints\SD3\sd3_medium_incl_clips.safetensors', 'hash': '080136ef'}, 'additional_modules': ['G:\ComfyUI\models\vae\t5xxl_fp8_e4m3fn.safetensors'], 'unet_storage_dtype': torch.float8_e4m3fn} Model selected: {'checkpoint_info': {'filename': 'G:\ComfyUI\models\checkpoints\SD3\sd3_medium_incl_clips.safetensors', 'hash': '080136ef'}, 'additional_modules': ['G:\ComfyUI\models\vae\t5xxl_fp8_e4m3fn.safetensors'], 'unet_storage_dtype': torch.float8_e4m3fn} Loading Model: {'checkpoint_info': {'filename': 'G:\ComfyUI\models\checkpoints\SD3\sd3_medium_incl_clips.safetensors', 'hash': '080136ef'}, 'additional_modules': ['G:\ComfyUI\models\vae\t5xxl_fp8_e4m3fn.safetensors'], 'unet_storage_dtype': torch.float8_e4m3fn} StateDict Keys: {'unet': 491, 'vae': 244, 'text_encoder': 196, 'text_encoder_2': 517, 'text_encoder_3': 220, 'ignore': 0} Using Detected T5 Data Type: torch.float8_e4m3fn Traceback (most recent call last): File "G:\webui_forge_cu121_torch231\webui\modules_forge\main_thread.py", line 30, in work self.result = self.func(*self.args, self.kwargs) File "G:\webui_forge_cu121_torch231\webui\modules\txt2img.py", line 110, in txt2img_function processed = processing.process_images(p) File "G:\webui_forge_cu121_torch231\webui\modules\processing.py", line 789, in process_images p.sd_model, just_reloaded = forge_model_reload() File "C:\Program Files\Python310\lib\site-packages\torch\utils_contextlib.py", line 115, in decorate_context return func(*args, *kwargs) File "G:\webui_forge_cu121_torch231\webui\modules\sd_models.py", line 501, in forge_model_reload sd_model = forge_loader(state_dict, additional_state_dicts=additional_state_dicts) File "C:\Program Files\Python310\lib\site-packages\torch\utils_contextlib.py", line 115, in decorate_context return func(args, kwargs) File "G:\webui_forge_cu121_torch231\webui\backend\loader.py", line 261, in forge_loader component = load_huggingface_component(estimated_config, component_name, lib_name, cls_name, local_path, component_sd) File "G:\webui_forge_cu121_torch231\webui\backend\loader.py", line 94, in load_huggingface_component model = IntegratedT5(config) File "G:\webui_forge_cu121_torch231\webui\backend\nn\t5.py", line 211, in init self.transformer = T5(config) File "G:\webui_forge_cu121_torch231\webui\backend\nn\t5.py", line 199, in init self.encoder = T5Stack(self.num_layers, model_dim, model_dim, config["d_ff"], config["dense_act_fn"], config["is_gated_act"], config["num_heads"], config["model_type"] != "umt5") File "G:\webui_forge_cu121_torch231\webui\backend\nn\t5.py", line 172, in init [T5Block(model_dim, inner_dim, ff_dim, ff_activation, gated_act, num_heads, relative_attention_bias=((not relative_attention) or (i == 0))) for i in range(num_layers)] File "G:\webui_forge_cu121_torch231\webui\backend\nn\t5.py", line 172, in