[Bug]: Deforum tab is not working after the last flux update

[EffyDark]

What happened and why was it ungood?

The Deforum tab was there definitely pre-update. Now is gone. No idea how to proceed on this one, just reporting. Reinstalled a couple of times to no avail.

[cgrossi]

Yea I've got this as well. I'm sure there are quite a few extensions that aren't regularly maintained that broke from this big update. I hope we can get a fix 🤞

[wolframfreiling]

I've got the same problem!

[yorama55]

Same here. Thanks.

---

*** Error loading script: deforum_api.py Traceback (most recent call last): File "E:\Forge Flux\webui\modules\scripts.py", line 525, in load_scripts script_module = script_loading.load_module(scriptfile.path) File "E:\Forge Flux\webui\modules\script_loading.py", line 13, in load_module module_spec.loader.exec_module(module) File "", line 883, in exec_module File "", line 241, in _call_with_frames_removed File "E:\Forge Flux\webui\extensions\sd-forge-deforum\scripts\deforum_api.py", line 29, in from deforum_api_models import Batch, DeforumJobErrorType, DeforumJobStatusCategory, DeforumJobPhase, DeforumJobStatus ModuleNotFoundError: No module named 'deforum_api_models'

[AIdud4212]

I have the same problem. After the Forge update, the Deforum tab disappeared

[kristapscocoo]

Having the same problem here.

[S4f3tyMarc]

Deforum won't work with the forge update as the new forge uses Gradio-4. So it will need to be updated to work with Gradio-4 first.

[wjmcool]

I simply fixed this bug and hope it's useful to you. Please copy the zip file to the installation directory and unzip it. enjoy:) sd-forge-deforum_patch.zip

[yorama55]

Deforum tab is now working with the patch. Thank you! All works OK when using SDXL models.

However using Flux models with Deforum produces black frames. I tried a few Flux models and most of them do not even work with Forge Txt2img. Two models do work well in Txt2img, flux1-dev-bnb-nf4-v2.safetensors and atomixFluxNF4_v10.safetensors. But as stated before, only black frames are generated when using Deforum with the same models. I tried different settings without any success. Using DPM++ 2M produced some noisy and blurry result but nothing useful. Thanks again.

In addition: For this testing I installed a fresh version of Forge using the all-in-one installer, after installing Deforum with Git locally in the extensions folder (using cmd) and the Deforum zip patch, the Deforum tab became available in Forge. But after I finished the testing (without any useful results), I then decided to update Forge to the latest version using update.bat. After that the Deforum tab in Forge disappeared again.

** Error loading script: deforum.py Traceback (most recent call last): File "E:\ForgeFlux\webui\modules\scripts.py", line 525, in load_scripts script_module = script_loading.load_module(scriptfile.path) File "E:\ForgeFlux\webui\modules\script_loading.py", line 13, in load_module module_spec.loader.exec_module(module) File "", line 883, in exec_module File "", line 241, in _call_with_frames_removed File "E:\ForgeFlux\webui\extensions\sd-forge-deforum\scripts\deforum.py", line 41, in init_deforum() File "E:\ForgeFlux\webui\extensions\sd-forge-deforum\scripts\deforum.py", line 33, in init_deforum from deforum_helpers.ui_right import on_ui_tabs File "E:\ForgeFlux\webui\extensions\sd-forge-deforum\scripts\deforum_helpers\ui_right.py", line 17, in from .args import DeforumOutputArgs, get_component_names, get_settings_component_names File "E:\ForgeFlux\webui\extensions\sd-forge-deforum\scripts\deforum_helpers\args.py", line 26, in from .deforum_controlnet import controlnet_component_names File "E:\ForgeFlux\webui\extensions\sd-forge-deforum\scripts\deforum_helpers\deforum_controlnet.py", line 29, in from .general_utils import count_files_in_folder, clean_gradio_path_strings # TODO: do it another way File "E:\ForgeFlux\webui\extensions\sd-forge-deforum\scripts\deforum_helpers\general_utils.py", line 21, in from basicsr.utils.download_util import load_file_from_url File "E:\ForgeFlux\system\python\lib\site-packages\basicsr__init__.py", line 4, in from .data import File "E:\ForgeFlux\system\python\lib\site-packages\basicsr\data__init.py", line 22, in _dataset_modules = [importlib.import_module(f'basicsr.data.{file_name}') for file_name in dataset_filenames] File "E:\ForgeFlux\system\python\lib\site-packages\basicsr\data\init.py", line 22, in _dataset_modules = [importlib.import_module(f'basicsr.data.{file_name}') for file_name in dataset_filenames] File "importlib\init__.py", line 126, in import_module File "E:\ForgeFlux\system\python\lib\site-packages\basicsr\data\realesrgan_dataset.py", line 11, in from basicsr.data.degradations import circular_lowpass_kernel, random_mixed_kernels File "E:\ForgeFlux\system\python\lib\site-packages\basicsr\data\degradations.py", line 8, in from torchvision.transforms.functional_tensor import rgb_to_grayscale ModuleNotFoundError: No module named 'torchvision.transforms.functional_tensor'

---

*** Error loading script: deforum_api.py Traceback (most recent call last): File "E:\ForgeFlux\webui\modules\scripts.py", line 525, in load_scripts script_module = script_loading.load_module(scriptfile.path) File "E:\ForgeFlux\webui\modules\script_loading.py", line 13, in load_module module_spec.loader.exec_module(module) File "", line 883, in exec_module File "", line 241, in _call_with_frames_removed File "E:\ForgeFlux\webui\extensions\sd-forge-deforum\scripts\deforum_api.py", line 29, in from deforum_api_models import Batch, DeforumJobErrorType, DeforumJobStatusCategory, DeforumJobPhase, DeforumJobStatus ModuleNotFoundError: No module named 'deforum_api_models'

[renatodigiorgio]

same here. deforum tab back after patch, but renders now black frames =/

[moebiussurfing]

not working here neither after the patch. missing deforum tab... i am getting these errors when starting:

PS D:\_AI\ComfyUI_windows_portable> D:\_AI\webui_forge_cu121_torch231\run.bat
'environment.bat' is not recognized as an internal or external command,
operable program or batch file.
venv "D:\_AI\webui_forge_cu121_torch231\webui\venv\Scripts\Python.exe"
Python 3.10.11 (tags/v3.10.11:7d4cc5a, Apr  5 2023, 00:38:17) [MSC v.1929 64 bit (AMD64)]
Version: f2.0.1v1.10.1-previous-450-gdb6448df
Commit hash: db6448df744a8929d8ac05d87dec9e85ec8ea48b
Launching Web UI with arguments: --listen
Total VRAM 24576 MB, total RAM 65445 MB
pytorch version: 2.3.1+cu121
Set vram state to: NORMAL_VRAM
Device: cuda:0 NVIDIA GeForce RTX 3090 : native
Hint: your device supports --cuda-malloc for potential speed improvements.
VAE dtype preferences: [torch.bfloat16, torch.float32] -> torch.bfloat16
CUDA Using Stream: False
Using pytorch cross attention
Using pytorch attention for VAE
reading metadata for D:\_AI\webui_forge_cu121_torch231\webui\models\Stable-diffusion\SD\MOHAWK_v18VAEBaked.safetensors: AssertionError
Traceback (most recent call last):
  File "D:\_AI\webui_forge_cu121_torch231\webui\modules\sd_models.py", line 89, in __init__
    self.metadata = cache.cached_data_for_file('safetensors-metadata', "checkpoint/" + name, filename, read_metadata)
  File "D:\_AI\webui_forge_cu121_torch231\webui\modules\cache.py", line 114, in cached_data_for_file
    value = func()
  File "D:\_AI\webui_forge_cu121_torch231\webui\modules\sd_models.py", line 81, in read_metadata
    metadata = read_metadata_from_safetensors(filename)
  File "D:\_AI\webui_forge_cu121_torch231\webui\modules\sd_models.py", line 270, in read_metadata_from_safetensors
    assert metadata_len > 2 and json_start in (b'{"', b"{'"), f"{filename} is not a safetensors file"
AssertionError: D:\_AI\webui_forge_cu121_torch231\webui\models\Stable-diffusion\SD\MOHAWK_v18VAEBaked.safetensors is not a safetensors file

ControlNet preprocessor location: D:\_AI\webui_forge_cu121_torch231\webui\models\ControlNetPreprocessor
*** Error loading script: deforum.py
    Traceback (most recent call last):
      File "D:\_AI\webui_forge_cu121_torch231\webui\modules\scripts.py", line 525, in load_scripts
        script_module = script_loading.load_module(scriptfile.path)
      File "D:\_AI\webui_forge_cu121_torch231\webui\modules\script_loading.py", line 13, in load_module
        module_spec.loader.exec_module(module)
      File "<frozen importlib._bootstrap_external>", line 883, in exec_module
      File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
      File "D:\_AI\webui_forge_cu121_torch231\webui\extensions\sd-forge-deforum\scripts\deforum.py", line 41, in <module>
        init_deforum()
      File "D:\_AI\webui_forge_cu121_torch231\webui\extensions\sd-forge-deforum\scripts\deforum.py", line 33, in init_deforum
        from deforum_helpers.ui_right import on_ui_tabs
      File "D:\_AI\webui_forge_cu121_torch231\webui\extensions\sd-forge-deforum\scripts\deforum_helpers\ui_right.py", line 17, in <module>
        from .args import DeforumOutputArgs, get_component_names, get_settings_component_names
      File "D:\_AI\webui_forge_cu121_torch231\webui\extensions\sd-forge-deforum\scripts\deforum_helpers\args.py", line 26, in <module>
        from .deforum_controlnet import controlnet_component_names
      File "D:\_AI\webui_forge_cu121_torch231\webui\extensions\sd-forge-deforum\scripts\deforum_helpers\deforum_controlnet.py", line 29, in <module>
        from .general_utils import count_files_in_folder, clean_gradio_path_strings  # TODO: do it another way
      File "D:\_AI\webui_forge_cu121_torch231\webui\extensions\sd-forge-deforum\scripts\deforum_helpers\general_utils.py", line 21, in <module>
        from basicsr.utils.download_util import load_file_from_url
    ModuleNotFoundError: No module named 'basicsr'

---
*** Error loading script: deforum_api.py
    Traceback (most recent call last):
      File "D:\_AI\webui_forge_cu121_torch231\webui\modules\scripts.py", line 525, in load_scripts
        script_module = script_loading.load_module(scriptfile.path)
      File "D:\_AI\webui_forge_cu121_torch231\webui\modules\script_loading.py", line 13, in load_module
        module_spec.loader.exec_module(module)
      File "<frozen importlib._bootstrap_external>", line 883, in exec_module
      File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
      File "D:\_AI\webui_forge_cu121_torch231\webui\extensions\sd-forge-deforum\scripts\deforum_api.py", line 29, in <module>
        from deforum_api_models import Batch, DeforumJobErrorType, DeforumJobStatusCategory, DeforumJobPhase, DeforumJobStatus
    ModuleNotFoundError: No module named 'deforum_api_models'

---
[Vec. CC] Style Sheet Loaded...
2024-08-30 18:23:46,064 - ControlNet - INFO - ControlNet UI callback registered.
reading metadata for D:\_AI\webui_forge_cu121_torch231\webui\models\Stable-diffusion\SD\MOHAWK_v18VAEBaked.safetensors: AssertionError
Traceback (most recent call last):
  File "D:\_AI\webui_forge_cu121_torch231\webui\modules\sd_models.py", line 89, in __init__
    self.metadata = cache.cached_data_for_file('safetensors-metadata', "checkpoint/" + name, filename, read_metadata)
  File "D:\_AI\webui_forge_cu121_torch231\webui\modules\cache.py", line 114, in cached_data_for_file
    value = func()
  File "D:\_AI\webui_forge_cu121_torch231\webui\modules\sd_models.py", line 81, in read_metadata
    metadata = read_metadata_from_safetensors(filename)
  File "D:\_AI\webui_forge_cu121_torch231\webui\modules\sd_models.py", line 270, in read_metadata_from_safetensors
    assert metadata_len > 2 and json_start in (b'{"', b"{'"), f"{filename} is not a safetensors file"
AssertionError: D:\_AI\webui_forge_cu121_torch231\webui\models\Stable-diffusion\SD\MOHAWK_v18VAEBaked.safetensors is not a safetensors file

Model selected: {'checkpoint_info': {'filename': 'D:\\_AI\\webui_forge_cu121_torch231\\webui\\models\\Stable-diffusion\\FLUX\\flux1-dev-fp8.safetensors', 'hash': 'be9881f4'}, 'additional_modules': ['D:\\_AI\\webui_forge_cu121_torch231\\webui\\models\\text_encoder\\clip_l.safetensors', 'D:\\_AI\\webui_forge_cu121_torch231\\webui\\models\\VAE\\ae.safetensors'], 'unet_storage_dtype': None}
Using online LoRAs in FP16: False
Running on local URL:  http://0.0.0.0:7860

To create a public link, set `share=True` in `launch()`.
Startup time: 20.5s (prepare environment: 2.8s, import torch: 7.2s, initialize shared: 0.2s, other imports: 0.6s, load scripts: 2.1s, create ui: 2.2s, gradio launch: 5.2s).
reading metadata for D:\_AI\webui_forge_cu121_torch231\webui\models\Stable-diffusion\SD\MOHAWK_v18VAEBaked.safetensors: AssertionError
Traceback (most recent call last):
  File "D:\_AI\webui_forge_cu121_torch231\webui\modules\sd_models.py", line 89, in __init__
    self.metadata = cache.cached_data_for_file('safetensors-metadata', "checkpoint/" + name, filename, read_metadata)
  File "D:\_AI\webui_forge_cu121_torch231\webui\modules\cache.py", line 114, in cached_data_for_file
    value = func()
  File "D:\_AI\webui_forge_cu121_torch231\webui\modules\sd_models.py", line 81, in read_metadata
    metadata = read_metadata_from_safetensors(filename)
  File "D:\_AI\webui_forge_cu121_torch231\webui\modules\sd_models.py", line 270, in read_metadata_from_safetensors
    assert metadata_len > 2 and json_start in (b'{"', b"{'"), f"{filename} is not a safetensors file"
AssertionError: D:\_AI\webui_forge_cu121_torch231\webui\models\Stable-diffusion\SD\MOHAWK_v18VAEBaked.safetensors is not a safetensors file

Environment vars changed: {'stream': False, 'inference_memory': 1024.0, 'pin_shared_memory': False}
------------------
[Low VRAM Warning] You just set Forge to use 100% GPU memory (23551.00 MB) to load model weights.
[Low VRAM Warning] This means you will have 0% GPU memory (0.00 MB) to do matrix computation. Computations may fallback to CPU or go Out of Memory.
[Low VRAM Warning] In many cases, image generation will be 10x slower.
[Low VRAM Warning] Make sure that you know what you are testing.
------------------

[droide]

https://discord.com/channels/1011314943380959313/1218711613524410428 https://github.com/Tok/sd-forge-deforum

[yorama55]

Thank you for all your efforts.

I can not get the Tok version to work. Installs OK without any errors and the Deforum tab exists. But whatever I do the first image frame looks good and then it immediately fizzles out into noise on the second frame. Tried various settings, disabled the Init, changed the default Deforum prompts and such in order to simplify as much as possible but at no avail.

Also, you can not save settings, practically you can and there is no error in the console when saving, and the settings file appears in the main folder. But if you re launch Forge and load the same saved settings into Deforum (without errors), and then change the name and save it again, this error always appears in the console and in some cases the console even crashes:

reading custom settings from E:\ForgeFlux\webui\deforum_settings 02.txt Traceback (most recent call last): File "E:\ForgeFlux\webui\modules\call_queue.py", line 76, in f res = list(func(*args, *kwargs)) File "E:\ForgeFlux\webui\modules\call_queue.py", line 55, in f res = func(args, **kwargs) File "E:\ForgeFlux\webui\extensions\sd-forge-deforum\scripts\deforum_helpers\settings.py", line 100, in save_settings filtered_combined["sd_model_name"] = sh.sd_model.sd_checkpoint_info.name AttributeError: 'FakeInitialModel' object has no attribute 'sd_checkpoint_info' 'FakeInitialModel' object has no attribute 'sd_checkpoint_info'

[droide]

For solve save setting problem try to delete env folder... I have same problem but i reinstalled all sd forge...

Il sab 31 ago 2024, 23:08 yorama55 @.***> ha scritto:

Thank you for all your efforts.

I can not get the Tok version to work. Installs OK without any errors and the Deforum tab exists. But whatever I do the first image frame looks good and then it immediately fizzles out into noise on the second frame. Tried various settings, disabled the Init, changed the default Deforum prompts and such in order to simplify as much as possible but at no avail.

Also, you can not save settings, practically you can and there is no error in the console when saving, and the settings file appears in the main folder. But if you re launch Forge and load the same saves settings into Deforum (without errors), and then change the name and save it again, this error always appears in the console and in some cases the console even crashes:

reading custom settings from E:\ForgeFlux\webui\deforum_settings 02.txt Traceback (most recent call last): File "E:\ForgeFlux\webui\modules\call_queue.py", line 76, in f res = list(func(*args, *kwargs)) File "E:\ForgeFlux\webui\modules\call_queue.py", line 55, in f res = func(args, **kwargs) File "E:\ForgeFlux\webui\extensions\sd-forge-deforum\scripts\deforum_helpers\settings.py", line 100, in save_settings filtered_combined["sd_model_name"] = sh.sd_model.sd_checkpoint_info.name AttributeError: 'FakeInitialModel' object has no attribute 'sd_checkpoint_info' 'FakeInitialModel' object has no attribute 'sd_checkpoint_info'

— Reply to this email directly, view it on GitHub https://github.com/deforum-art/sd-forge-deforum/issues/24#issuecomment-2323043487, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABDEB3X47S5UINXABSUROZTZUIWD7AVCNFSM6AAAAABMPQVZN2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDGMRTGA2DGNBYG4 . You are receiving this because you commented.Message ID: @.***>

[yorama55]

Some progress with the Tok version but still nothing useful. Using any of the new keyframe distribution options other than Off crashes the rendering after a few seconds (3070Ti). Off does not crash but the first frame is always great (actually real good), however within a couple of seconds the images wash out and go completely abstract. Also Deforum doesn't seem to follow the prompt changes for creating new scenes within the rendering. Tried and played around with a bunch of different settings at no avail. The Deforum settings-save issue looks like it is fixed. Thank you.

[yorama55]

First frame:

https://i.postimg.cc/DzSccMgN/20240901144020-000000000.png

A couple of seconds later:

https://i.postimg.cc/qqqBt202/20240901144020-000000046.png

Relevant settings:

"W": 1344, "H": 768, "show_info_on_ui": true, "tiling": false, "restore_faces": false, "seed_resize_from_w": 0, "seed_resize_from_h": 0, "seed": -1, "sampler": "Euler", "scheduler": "Simple", "steps": 20, "batch_name": "test 07", "seed_behavior": "iter", "seed_iter_N": 3, "use_init": false, "strength": 0.85, "strength_0_no_init": false, "prompts": { "0": "A towering ancient tree creature resembling a blend of an oak and a humanoid, with glowing blue veins of energy running through its bark, Its eyes are glowing and vibrant green, filled with wisdom and ancient power, Surrounding it is a lush enchanted forest with bioluminescent plants and flowers with floating orbs of colorful light, The atmosphere is misty and magical with rays of sunlight piercing through the dense canopy", "148": "A fierce armored knight wielding a glowing sword, engaged in a battle with a massive dragon amidst a thunderstorm, The dragon has shimmering black scales and its eyes glow an ominous red, Lightning crackles in the sky and illuminating the scene with bursts of light, The ground is a rocky terrain with patches of fire and rain pours down heavily adding a dramatic effect to the intense battle", "296": "A majestic city built on a series of floating islands connected by glowing bridges of energy, The architecture is a mix of Gothic and steampunk with tall spires, ornate buildings and large windmills, Below the city are layers of thick fluffy clouds tinged with sunset colors, In the sky are flying ships and air balloons navigating the skies, adding a sense of wonder and movement to the scene", "452": "A powerful sorceress with long flowing silver hair and a dark intricate robe adorned with mystical symbols, She stands in a ruined temple with ancient stone pillars, casting a spell that opens a swirling glowing portal filled with stars and cosmic energy, The ground is littered with runes and magical artifacts, and wisps of purple and blue magic flow around her, giving a sense of dynamic power" }, "animation_prompts_positive": "", "animation_prompts_negative": "", "animation_mode": "2D", "max_frames": 648, "border": "replicate", "angle": "0: (0)", "zoom": "0: (1.004)", "noise_schedule": "0: (0.08)", "strength_schedule": "0: (0.72)", "keyframe_strength_schedule": "0: (0)", "contrast_schedule": "0: (1.0)", "cfg_scale_schedule": "0: (1.0)", "distilled_cfg_scale_schedule": "0: (3.5)", "enable_steps_scheduling": false, "steps_schedule": "0: (20)", "keyframe_distribution": "Off", "diffusion_cadence": 1, "fps": 24,

[yorama55]

Still no useful results using the Tok version with Flux, even though I am now able to use the experimental rendering options without crashing, same results. However with the latest Forge and Tok commits Deforum now works with SDXL models. As you can see in this short test run (wowXLPDFlux_wowXL7.safetensors). Thank You! Anticipating with patience that it will soon work with Flux.

https://youtu.be/hqH3eRAji5k

[aklevecz]

@yorama55 I am having the same issues with my images degrading into weird abstract color. I'm not smart enough to have a clear diagnosis or solution, but I think it has something to do with too much noise accumulating over time, which is why it has that particular multi-colored pattern. I'm going to try and play with the noise settings and see if I have any luck.