[Feature Request]: Support Stable Cascade

huchenlei commented 4 months ago

Is there an existing issue for this?

[X] I have searched the existing issues and checked the recent builds/commits

What would your feature do ?

https://github.com/Stability-AI/StableCascade

Proposed workflow

Stable cascade should be a selectable model option

Additional information

Probably need separate slider for steps in "Stage C" and "Stage A&B"

aolko commented 4 months ago

Will it work on cards with <20gb VRAM?

E2GO commented 4 months ago

Will it work on cards with <20gb VRAM?

You can test yourself already: https://www.reddit.com/r/StableDiffusion/comments/1apy55i/stable_cascade_oneclick_installer/

CCpt5 commented 4 months ago

This works for Comfy:: https://github.com/kijai/ComfyUI-DiffusersStableCascade?tab=readme-ov-file

Hope y'all get it hooked up in Forge soon!

strawberrymelonpanda commented 4 months ago

This works for Comfy:: https://github.com/kijai/ComfyUI-DiffusersStableCascade?tab=readme-ov-file

Key words in that repo are, as is often with just released things in the "research" phase, "using diffusers". Auto doesn't use Diffusers, and Comfy doesn't either, but has an unsupported Diffusers node that makes things like the above easier.

@huchenlei Is wrapping a Diffusers model something Forge can do? That'd be neat for rapid integration, but otherwise it seems like upstream support in Auto's repo would be best.

strawberrymelonpanda commented 4 months ago

As far as additional model support goes, if going that route then the whole list of models from SD.Next's supported list would honestly be interesting and a good long-term goal.

Model support

Additional models will be added as they become available and there is public interest in them

aMUSEd 256 256 and 512
Segmind Vega
Segmind SSD-1B
Segmind SegMoE SD and SD-XL
Kandinsky 2.1 and 2.2 and latest 3.0
PixArt-α XL 2 Medium and Large
Warp Wuerstchen
Playground v1, v2 256, v2 512, v2 1024
Tsinghua UniDiffusion
DeepFloyd IF Medium and Large
ModelScope T2V
Segmind SD Distilled (all variants)
BLIP-Diffusion

CCpt5 commented 4 months ago

Temporary solution shared on twitter: https://twitter.com/blue_pen5805/status/1757622466858635419

https://github.com/blue-pen5805/sdweb-easy-stablecascade-diffusers

Extension to allow for generation. Dev tweeted just after to delete extension after Cascade support is added natively.

mcDandy commented 4 months ago

That extension however does crash for me on both A1111 and forge.

*** Error loading script: easy_stablecascade_diffusers.py
    Traceback (most recent call last):
      File "D:\Users\danda\Downloads\sdwebui\webui\modules\scripts.py", line 544, in load_scripts
        script_module = script_loading.load_module(scriptfile.path)
      File "D:\Users\danda\Downloads\sdwebui\webui\modules\script_loading.py", line 10, in load_module
        module_spec.loader.exec_module(module)
      File "<frozen importlib._bootstrap_external>", line 883, in exec_module
      File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
      File "D:\Users\danda\Downloads\sdwebui\webui\extensions\sdweb-easy-stablecascade-diffusers\scripts\easy_stablecascade_diffusers.py", line 5, in <module>
        from diffusers import StableCascadeDecoderPipeline, StableCascadePriorPipeline
    ImportError: cannot import name 'StableCascadeDecoderPipeline' from 'diffusers' (D:\Users\danda\Downloads\sdwebui\system\python\lib\site-packages\diffusers\__init__.py)

---

ResidentChief commented 4 months ago

That extension however does crash for me on both A1111 and forge.


*** Error loading script: easy_stablecascade_diffusers.py
    Traceback (most recent call last):
      File "D:\Users\danda\Downloads\sdwebui\webui\modules\scripts.py", line 544, in load_scripts
        script_module = script_loading.load_module(scriptfile.path)
      File "D:\Users\danda\Downloads\sdwebui\webui\modules\script_loading.py", line 10, in load_module
        module_spec.loader.exec_module(module)
      File "<frozen importlib._bootstrap_external>", line 883, in exec_module
      File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
      File "D:\Users\danda\Downloads\sdwebui\webui\extensions\sdweb-easy-stablecascade-diffusers\scripts\easy_stablecascade_diffusers.py", line 5, in <module>
        from diffusers import StableCascadeDecoderPipeline, StableCascadePriorPipeline
    ImportError: cannot import name 'StableCascadeDecoderPipeline' from 'diffusers' (D:\Users\danda\Downloads\sdwebui\system\python\lib\site-packages\diffusers\__init__.py)

---

Are you running the latest version of diffusers?

mcDandy commented 4 months ago

Tried updating to 0.26.3 from 0.25.0. That did not help. It is trying to downloat it's own difussers or something. I do not understand the install error log. Thank you however.

Python 3.10.6 (tags/v3.10.6:9c7b4bd, Aug  1 2022, 21:53:49) [MSC v.1932 64 bit (AMD64)]
Version: f0.0.14v1.8.0rc-latest-184-g43c9e3b5c
Commit hash: 43c9e3b5ce1642073c7a9684e36b45489eeb4a49
Installing requirements
*** Error running install.py for extension D:\Users\danda\Downloads\sdwebui\webui\extensions\sdweb-easy-stablecascade-diffusers.
*** Command: "D:\Users\danda\Downloads\sdwebui\system\python\python.exe" "D:\Users\danda\Downloads\sdwebui\webui\extensions\sdweb-easy-stablecascade-diffusers\install.py"
*** Error code: 1
*** stdout: Installing diffusers@wuerstchen-v3
***
*** stderr: Traceback (most recent call last):
***   File "D:\Users\danda\Downloads\sdwebui\webui\extensions\sdweb-easy-stablecascade-diffusers\install.py", line 4, in <module>
***     from diffusers import StableCascadeDecoderPipeline
*** ImportError: cannot import name 'StableCascadeDecoderPipeline' from 'diffusers' (D:\Users\danda\Downloads\sdwebui\system\python\lib\site-packages\diffusers\__init__.py)
***
*** During handling of the above exception, another exception occurred:
***
*** Traceback (most recent call last):
***   File "D:\Users\danda\Downloads\sdwebui\webui\extensions\sdweb-easy-stablecascade-diffusers\install.py", line 6, in <module>
***     launch.run_pip(f"install git+https://github.com/kashif/diffusers.git@a3dc21385b7386beb3dab3a9845962ede6765887", "diffusers@wuerstchen-v3")
***   File "D:\Users\danda\Downloads\sdwebui\webui\modules\launch_utils.py", line 152, in run_pip
***     return run(f'"{python}" -m pip {command} --prefer-binary{index_url_line}', desc=f"Installing {desc}", errdesc=f"Couldn't install {desc}", live=live)
***   File "D:\Users\danda\Downloads\sdwebui\webui\modules\launch_utils.py", line 124, in run
***     raise RuntimeError("\n".join(error_bits))
*** RuntimeError: Couldn't install diffusers@wuerstchen-v3.
*** Command: "D:\Users\danda\Downloads\sdwebui\system\python\python.exe" -m pip install git+https://github.com/kashif/diffusers.git@a3dc21385b7386beb3dab3a9845962ede6765887 --prefer-binary
*** Error code: 2
*** stdout: Collecting git+https://github.com/kashif/diffusers.git@a3dc21385b7386beb3dab3a9845962ede6765887
***   Cloning https://github.com/kashif/diffusers.git (to revision a3dc21385b7386beb3dab3a9845962ede6765887) to c:\users\danda\appdata\local\temp\pip-req-build-0u_wfpgr
***   Resolved https://github.com/kashif/diffusers.git to commit a3dc21385b7386beb3dab3a9845962ede6765887
***   Installing build dependencies: started
***   Installing build dependencies: finished with status 'done'
***   Getting requirements to build wheel: started
***   Getting requirements to build wheel: finished with status 'done'
***
*** stderr:   Running command git clone --filter=blob:none --quiet https://github.com/kashif/diffusers.git 'C:\Users\danda\AppData\Local\Temp\pip-req-build-0u_wfpgr'
***   Running command git rev-parse -q --verify 'sha^a3dc21385b7386beb3dab3a9845962ede6765887'
***   Running command git fetch -q https://github.com/kashif/diffusers.git a3dc21385b7386beb3dab3a9845962ede6765887
***   Running command git checkout -q a3dc21385b7386beb3dab3a9845962ede6765887
*** ERROR: Exception:
*** Traceback (most recent call last):
***   File "D:\Users\danda\Downloads\sdwebui\system\python\lib\site-packages\pip\_internal\cli\base_command.py", line 180, in exc_logging_wrapper
***     status = run_func(*args)
***   File "D:\Users\danda\Downloads\sdwebui\system\python\lib\site-packages\pip\_internal\cli\req_command.py", line 245, in wrapper
***     return func(self, options, args)
***   File "D:\Users\danda\Downloads\sdwebui\system\python\lib\site-packages\pip\_internal\commands\install.py", line 377, in run
***     requirement_set = resolver.resolve(
***   File "D:\Users\danda\Downloads\sdwebui\system\python\lib\site-packages\pip\_internal\resolution\resolvelib\resolver.py", line 76, in resolve
***     collected = self.factory.collect_root_requirements(root_reqs)
***   File "D:\Users\danda\Downloads\sdwebui\system\python\lib\site-packages\pip\_internal\resolution\resolvelib\factory.py", line 534, in collect_root_requirements
***     reqs = list(
***   File "D:\Users\danda\Downloads\sdwebui\system\python\lib\site-packages\pip\_internal\resolution\resolvelib\factory.py", line 490, in _make_requirements_from_install_req
***     cand = self._make_base_candidate_from_link(
***   File "D:\Users\danda\Downloads\sdwebui\system\python\lib\site-packages\pip\_internal\resolution\resolvelib\factory.py", line 228, in _make_base_candidate_from_link
***     self._link_candidate_cache[link] = LinkCandidate(
***   File "D:\Users\danda\Downloads\sdwebui\system\python\lib\site-packages\pip\_internal\resolution\resolvelib\candidates.py", line 290, in __init__
***     super().__init__(
***   File "D:\Users\danda\Downloads\sdwebui\system\python\lib\site-packages\pip\_internal\resolution\resolvelib\candidates.py", line 156, in __init__
***     self.dist = self._prepare()
***   File "D:\Users\danda\Downloads\sdwebui\system\python\lib\site-packages\pip\_internal\resolution\resolvelib\candidates.py", line 222, in _prepare
***     dist = self._prepare_distribution()
***   File "D:\Users\danda\Downloads\sdwebui\system\python\lib\site-packages\pip\_internal\resolution\resolvelib\candidates.py", line 301, in _prepare_distribution
***     return preparer.prepare_linked_requirement(self._ireq, parallel_builds=True)
***   File "D:\Users\danda\Downloads\sdwebui\system\python\lib\site-packages\pip\_internal\operations\prepare.py", line 525, in prepare_linked_requirement
***     return self._prepare_linked_requirement(req, parallel_builds)
***   File "D:\Users\danda\Downloads\sdwebui\system\python\lib\site-packages\pip\_internal\operations\prepare.py", line 640, in _prepare_linked_requirement
***     dist = _get_prepared_distribution(
***   File "D:\Users\danda\Downloads\sdwebui\system\python\lib\site-packages\pip\_internal\operations\prepare.py", line 71, in _get_prepared_distribution
***     abstract_dist.prepare_distribution_metadata(
***   File "D:\Users\danda\Downloads\sdwebui\system\python\lib\site-packages\pip\_internal\distributions\sdist.py", line 54, in prepare_distribution_metadata
***     self._install_build_reqs(finder)
***   File "D:\Users\danda\Downloads\sdwebui\system\python\lib\site-packages\pip\_internal\distributions\sdist.py", line 124, in _install_build_reqs
***     build_reqs = self._get_build_requires_wheel()
***   File "D:\Users\danda\Downloads\sdwebui\system\python\lib\site-packages\pip\_internal\distributions\sdist.py", line 101, in _get_build_requires_wheel
***     return backend.get_requires_for_build_wheel()
***   File "D:\Users\danda\Downloads\sdwebui\system\python\lib\site-packages\pip\_internal\utils\misc.py", line 745, in get_requires_for_build_wheel
***     return super().get_requires_for_build_wheel(config_settings=cs)
***   File "D:\Users\danda\Downloads\sdwebui\system\python\lib\site-packages\pip\_vendor\pyproject_hooks\_impl.py", line 166, in get_requires_for_build_wheel
***     return self._call_hook('get_requires_for_build_wheel', {
***   File "D:\Users\danda\Downloads\sdwebui\system\python\lib\site-packages\pip\_vendor\pyproject_hooks\_impl.py", line 321, in _call_hook
***     raise BackendUnavailable(data.get('traceback', ''))
*** pip._vendor.pyproject_hooks._impl.BackendUnavailable: Traceback (most recent call last):
***   File "D:\Users\danda\Downloads\sdwebui\system\python\lib\site-packages\pip\_vendor\pyproject_hooks\_in_process\_in_process.py", line 77, in _build_backend
***     obj = import_module(mod_path)
***   File "importlib\__init__.py", line 126, in import_module
***   File "<frozen importlib._bootstrap>", line 1050, in _gcd_import
***   File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
***   File "<frozen importlib._bootstrap>", line 992, in _find_and_load_unlocked
***   File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
***   File "<frozen importlib._bootstrap>", line 1050, in _gcd_import
***   File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
***   File "<frozen importlib._bootstrap>", line 1006, in _find_and_load_unlocked
***   File "<frozen importlib._bootstrap>", line 688, in _load_unlocked
***   File "<frozen importlib._bootstrap_external>", line 883, in exec_module
***   File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
***   File "C:\Users\danda\AppData\Local\Temp\pip-build-env-u85shcmp\overlay\Lib\site-packages\setuptools\__init__.py", line 13, in <module>
***     from . import version as _version_module
***   File "C:\Users\danda\AppData\Local\Temp\pip-build-env-u85shcmp\overlay\Lib\site-packages\setuptools\version.py", line 1, in <module>
***     from ._importlib import metadata
***   File "C:\Users\danda\AppData\Local\Temp\pip-build-env-u85shcmp\overlay\Lib\site-packages\setuptools\_importlib.py", line 45, in <module>
***     import importlib.metadata as metadata  # noqa: F401
***   File "importlib\metadata\__init__.py", line 17, in <module>
***   File "importlib\metadata\_adapters.py", line 3, in <module>
***   File "email\message.py", line 15, in <module>
***   File "email\utils.py", line 29, in <module>
***   File "socket.py", line 51, in <module>
*** ModuleNotFoundError: No module named '_socket'
***
***

sdbds commented 4 months ago

https://github.com/blue-pen5805/sdweb-easy-stablecascade-diffusers sd-webui extension for stage-c

benjamin-bertram commented 4 months ago

This works for me https://github.com/benjamin-bertram/sdweb-easy-stablecascade-diffusers, its a fork of blue-pen with added apple support and borrowed the buttons from the ui to send images to different tabs

brentjohnston commented 4 months ago

If devs here add Cascade support, can you allow for selecting different stage c and clip text encoder model checkpoints? I'm having issues with the official comfyui workflow doing this (produces errors) and have to use comfy unet nodes to get the finetuned text encoder and stage c checkpoints to work.

Here is a thread covering this https://github.com/bmaltais/kohya_ss/issues/1982

obs945 commented 3 months ago

That extension however does crash for me on both A1111 and forge.

*** Error loading script: easy_stablecascade_diffusers.py
    Traceback (most recent call last):
      File "D:\Users\danda\Downloads\sdwebui\webui\modules\scripts.py", line 544, in load_scripts
        script_module = script_loading.load_module(scriptfile.path)
      File "D:\Users\danda\Downloads\sdwebui\webui\modules\script_loading.py", line 10, in load_module
        module_spec.loader.exec_module(module)
      File "<frozen importlib._bootstrap_external>", line 883, in exec_module
      File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
      File "D:\Users\danda\Downloads\sdwebui\webui\extensions\sdweb-easy-stablecascade-diffusers\scripts\easy_stablecascade_diffusers.py", line 5, in <module>
        from diffusers import StableCascadeDecoderPipeline, StableCascadePriorPipeline
    ImportError: cannot import name 'StableCascadeDecoderPipeline' from 'diffusers' (D:\Users\danda\Downloads\sdwebui\system\python\lib\site-packages\diffusers\__init__.py)

---

You may try this. This works for me. I also install https://github.com/benjamin-bertram/sdweb-easy-stablecascade-diffusers as stablecascade extension.

pip install git+https://github.com/kashif/diffusers.git@a3dc21385b7386beb3dab3a9845962ede6765887 --force

lllyasviel / stable-diffusion-webui-forge