"Flux Control Net models not working? Where to download, then?"

[LIQUIDMIND111]

hELLO, I TRIED USING CONTROL NET MODELS FROM THIS PAGE [MODEL WAS DOWNLAODED FROM https://huggingface.co/XLabs-AI/flux-controlnet-collections] - AND WHEN TRYING TO USE CONTROL NET ON FORGE, GOT THIS ERROR:

2024-08-17 21:38:29,461 - ControlNet - ERROR - Recognizing Control Model failed: C:\TEMP\FORGEFLUX\webui\models\ControlNet\flux-canny-controlnet_v2.safetensors

Error running process: C:\TEMP\FORGEFLUX\webui\extensions-builtin\sd_forge_controlnet\scripts\controlnet.py Traceback (most recent call last): File "C:\TEMP\FORGEFLUX\webui\modules\scripts.py", line 844, in process script.process(p, script_args) File "C:\TEMP\FORGEFLUX\system\python\lib\site-packages\torch\utils_contextlib.py", line 115, in decorate_context return func(*args, *kwargs) File "C:\TEMP\FORGEFLUX\webui\extensions-builtin\sd_forge_controlnet\scripts\controlnet.py", line 554, in process self.process_unit_after_click_generate(p, unit, params, args, kwargs) File "C:\TEMP\FORGEFLUX\system\python\lib\site-packages\torch\utils_contextlib.py", line 115, in decorate_context return func(*args, **kwargs) File "C:\TEMP\FORGEFLUX\webui\extensions-builtin\sd_forge_controlnet\scripts\controlnet.py", line 414, in process_unit_after_click_generate assert params.model is not None, logger.error(f"Recognizing Control Model failed: {model_filename}") AssertionError: None

MODEL WAS DOWNLAODED FROM https://huggingface.co/XLabs-AI/flux-controlnet-collections

[Tom-Neverwinter]

curious did you set the text and vae as well? they will all be in the vae folder

[LIQUIDMIND111]

Well for flux I’m using iiiyasviel models that actually work without addin all those extra models like clip and vae. I thought it was all I need since all the YouTube tutorials do not mention anything about downloading this extra models for forge ui.

So what do I need to do that I’m not doing?

On Sun, Aug 18, 2024 at 3:28 AM Tom-Neverwinter @.***> wrote:

curious did you set the text and vae as well? they will all be in the vae folder

— Reply to this email directly, view it on GitHub https://github.com/lllyasviel/stable-diffusion-webui-forge/issues/1253#issuecomment-2295159554, or unsubscribe https://github.com/notifications/unsubscribe-auth/A22MJBBYXEQRH6QIBIGOMPDZSBEIZAVCNFSM6AAAAABMV4XZWCVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDEOJVGE2TSNJVGQ . You are receiving this because you authored the thread.Message ID: @.*** com>

[fabfabsto]

I encounter also problems with controlnet and flux. I'm trying flux1-dev-fp8.safetensors with controlnet deactivated I can generate images, also did the sanity check and came out ok. I tried different depth models:

flux-depth-controlnet-v3 [add17cb7] from https://huggingface.co/XLabs-AI/flux-controlnet-depth-v3 also in Verion 2

I'm getting this error, an image is generated but controlnet was not used:

 Error running postprocess_batch_list: C:\webui_forge\webui\extensions-builtin\sd_forge_controlnet\scripts\controlnet.py
    Traceback (most recent call last):
      File "C:\webui_forge\webui\modules\scripts.py", line 916, in postprocess_batch_list
        script.postprocess_batch_list(p, pp, *script_args, **kwargs)
      File "C:\webui_forge\system\python\lib\site-packages\torch\utils\_contextlib.py", line 115, in decorate_context
        return func(*args, **kwargs)
      File "C:\webui_forge\webui\extensions-builtin\sd_forge_controlnet\scripts\controlnet.py", line 567, in postprocess_batch_list
        self.process_unit_after_every_sampling(p, unit, self.current_params[i], pp, *args, **kwargs)
    KeyError: 0

when I go back to old depth model (diffusers_xl_depth_full [2f51180b]) they throw a different error and no image is being generated:

Traceback (most recent call last):
  File "C:\webui_forge\webui\modules_forge\main_thread.py", line 30, in work
    self.result = self.func(*self.args, **self.kwargs)
  File "C:\webui_forge\webui\modules\txt2img.py", line 110, in txt2img_function
    processed = processing.process_images(p)
  File "C:\webui_forge\webui\modules\processing.py", line 813, in process_images
    res = process_images_inner(p)
  File "C:\webui_forge\webui\modules\processing.py", line 956, in process_images_inner
    samples_ddim = p.sample(conditioning=p.c, unconditional_conditioning=p.uc, seeds=p.seeds, subseeds=p.subseeds, subseed_strength=p.subseed_strength, prompts=p.prompts)
  File "C:\webui_forge\webui\modules\processing.py", line 1327, in sample
    samples = self.sampler.sample(self, x, conditioning, unconditional_conditioning, image_conditioning=self.txt2img_image_conditioning(x))
  File "C:\webui_forge\webui\modules\sd_samplers_kdiffusion.py", line 234, in sample
    samples = self.launch_sampling(steps, lambda: self.func(self.model_wrap_cfg, x, extra_args=self.sampler_extra_args, disable=False, callback=self.callback_state, **extra_params_kwargs))
  File "C:\webui_forge\webui\modules\sd_samplers_common.py", line 272, in launch_sampling
    return func()
  File "C:\webui_forge\webui\modules\sd_samplers_kdiffusion.py", line 234, in <lambda>
    samples = self.launch_sampling(steps, lambda: self.func(self.model_wrap_cfg, x, extra_args=self.sampler_extra_args, disable=False, callback=self.callback_state, **extra_params_kwargs))
  File "C:\webui_forge\system\python\lib\site-packages\torch\utils\_contextlib.py", line 115, in decorate_context
    return func(*args, **kwargs)
  File "C:\webui_forge\webui\k_diffusion\sampling.py", line 128, in sample_euler
    denoised = model(x, sigma_hat * s_in, **extra_args)
  File "C:\webui_forge\system\python\lib\site-packages\torch\nn\modules\module.py", line 1532, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "C:\webui_forge\system\python\lib\site-packages\torch\nn\modules\module.py", line 1541, in _call_impl
    return forward_call(*args, **kwargs)
  File "C:\webui_forge\webui\modules\sd_samplers_cfg_denoiser.py", line 186, in forward
    denoised, cond_pred, uncond_pred = sampling_function(self, denoiser_params=denoiser_params, cond_scale=cond_scale, cond_composition=cond_composition)
  File "C:\webui_forge\webui\backend\sampling\sampling_function.py", line 333, in sampling_function
    for h in cond + uncond:
TypeError: can only concatenate list (not "NoneType") to list
can only concatenate list (not "NoneType") to list

[ClipSkipper]

2024-08-18 20:41:00,320 - ControlNet - ERROR - Recognizing Control Model failed: C:\FORGE\webui\models\ControlNet\flux-depth-controlnet-v3.safetensors Error running process: C:\FORGE\webui\extensions-builtin\sd_forge_controlnet\scripts\controlnet.py Traceback (most recent call last): File "C:\FORGE\webui\modules\scripts.py", line 844, in process script.process(p, script_args) File "C:\FORGE\webui\venv\lib\site-packages\torch\utils_contextlib.py", line 115, in decorate_context return func(*args, *kwargs) File "C:\FORGE\webui\extensions-builtin\sd_forge_controlnet\scripts\controlnet.py", line 554, in process self.process_unit_after_click_generate(p, unit, params, args, kwargs) File "C:\FORGE\webui\venv\lib\site-packages\torch\utils_contextlib.py", line 115, in decorate_context return func(*args, **kwargs) File "C:\FORGE\webui\extensions-builtin\sd_forge_controlnet\scripts\controlnet.py", line 414, in process_unit_after_click_generate assert params.model is not None, logger.error(f"Recognizing Control Model failed: {model_filename}") AssertionError: None

[LIQUIDMIND111]

I encounter also problems with controlnet and flux. I'm trying flux1-dev-fp8.safetensors with controlnet deactivated I can generate images, also did the sanity check and came out ok. I tried different depth models:

flux-depth-controlnet-v3 [add17cb7] from https://huggingface.co/XLabs-AI/flux-controlnet-depth-v3 also in Verion 2

I'm getting this error, an image is generated but controlnet was not used:

 Error running postprocess_batch_list: C:\webui_forge\webui\extensions-builtin\sd_forge_controlnet\scripts\controlnet.py
    Traceback (most recent call last):
      File "C:\webui_forge\webui\modules\scripts.py", line 916, in postprocess_batch_list
        script.postprocess_batch_list(p, pp, *script_args, **kwargs)
      File "C:\webui_forge\system\python\lib\site-packages\torch\utils\_contextlib.py", line 115, in decorate_context
        return func(*args, **kwargs)
      File "C:\webui_forge\webui\extensions-builtin\sd_forge_controlnet\scripts\controlnet.py", line 567, in postprocess_batch_list
        self.process_unit_after_every_sampling(p, unit, self.current_params[i], pp, *args, **kwargs)
    KeyError: 0

when I go back to old depth model (diffusers_xl_depth_full [2f51180b]) they throw a different error and no image is being generated:

Traceback (most recent call last):
  File "C:\webui_forge\webui\modules_forge\main_thread.py", line 30, in work
    self.result = self.func(*self.args, **self.kwargs)
  File "C:\webui_forge\webui\modules\txt2img.py", line 110, in txt2img_function
    processed = processing.process_images(p)
  File "C:\webui_forge\webui\modules\processing.py", line 813, in process_images
    res = process_images_inner(p)
  File "C:\webui_forge\webui\modules\processing.py", line 956, in process_images_inner
    samples_ddim = p.sample(conditioning=p.c, unconditional_conditioning=p.uc, seeds=p.seeds, subseeds=p.subseeds, subseed_strength=p.subseed_strength, prompts=p.prompts)
  File "C:\webui_forge\webui\modules\processing.py", line 1327, in sample
    samples = self.sampler.sample(self, x, conditioning, unconditional_conditioning, image_conditioning=self.txt2img_image_conditioning(x))
  File "C:\webui_forge\webui\modules\sd_samplers_kdiffusion.py", line 234, in sample
    samples = self.launch_sampling(steps, lambda: self.func(self.model_wrap_cfg, x, extra_args=self.sampler_extra_args, disable=False, callback=self.callback_state, **extra_params_kwargs))
  File "C:\webui_forge\webui\modules\sd_samplers_common.py", line 272, in launch_sampling
    return func()
  File "C:\webui_forge\webui\modules\sd_samplers_kdiffusion.py", line 234, in <lambda>
    samples = self.launch_sampling(steps, lambda: self.func(self.model_wrap_cfg, x, extra_args=self.sampler_extra_args, disable=False, callback=self.callback_state, **extra_params_kwargs))
  File "C:\webui_forge\system\python\lib\site-packages\torch\utils\_contextlib.py", line 115, in decorate_context
    return func(*args, **kwargs)
  File "C:\webui_forge\webui\k_diffusion\sampling.py", line 128, in sample_euler
    denoised = model(x, sigma_hat * s_in, **extra_args)
  File "C:\webui_forge\system\python\lib\site-packages\torch\nn\modules\module.py", line 1532, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "C:\webui_forge\system\python\lib\site-packages\torch\nn\modules\module.py", line 1541, in _call_impl
    return forward_call(*args, **kwargs)
  File "C:\webui_forge\webui\modules\sd_samplers_cfg_denoiser.py", line 186, in forward
    denoised, cond_pred, uncond_pred = sampling_function(self, denoiser_params=denoiser_params, cond_scale=cond_scale, cond_composition=cond_composition)
  File "C:\webui_forge\webui\backend\sampling\sampling_function.py", line 333, in sampling_function
    for h in cond + uncond:
TypeError: can only concatenate list (not "NoneType") to list
can only concatenate list (not "NoneType") to list

yep, there is another post like this, but NO reply yet from @lllyasviel

[Vagrin]

I've yet to have a ControlNet work once so far. Like the original poster I got my CNs from: https://huggingface.co/XLabs-AI/flux-controlnet-collections/tree/main I keep trying them from time to time. It doesn't error out it just has no effect. ControlNets work for everyone else? If they're not working at all, that's fine. I can wait. I'm just so thrilled to play with Flux without needing RatsNestUI... oops... I mean ComfyUI. The relevant error I think: Error running process_before_every_sampling: D:\Auto1111\Forge\webui\extensions-builtin\sd_forge_controlnet\scripts\controlnet.py

[ClipSkipper]

I have found though it does generate an actual depth map alongside the output generation, but is giving zero guidance to the actual output.

[LIQUIDMIND111]

I have found though it does generate an actual depth map alongside the output generation, but is giving zero guidance to the actual output.

but can you get the depth map at least? does control net saves the maps?

[ClipSkipper]

I have found though it does generate an actual depth map alongside the output generation, but is giving zero guidance to the actual output.

but can you get the depth map at least? does control net saves the maps?

doesnt appear too just appears alongside in the preview window, but then that might be my actual settings, if theres a path to save them I might have it blank, could always open in browser and save it if I needed to, just weird that works but not the actual guidance.

[LIQUIDMIND111]

I have found though it does generate an actual depth map alongside the output generation, but is giving zero guidance to the actual output.

but can you get the depth map at least? does control net saves the maps?

doesnt appear too just appears alongside in the preview window, but then that might be my actual settings, if theres a path to save them I might have it blank, could always open in browser and save it if I needed to, just weird that works but not the actual guidance.

im using an online depth map generator for now until this works...

[mertayd0]

I am having the same issue, any updates?

[R2008KK]

Are there those who have controlnet working, like the user who suggested downgrading the onnx version?

[R2008KK]

As I found out, the addition of controlnet support was planned to be done in October, but the current capabilities of controlnet are still limited and are not very suitable for flux, so it is delayed. However, I do not know where this information comes from, so I do not know how true it is.