search

deforum-art / sd-webui-deforum

Deforum extension for AUTOMATIC1111's Stable Diffusion webui
https://deforum.github.io
Other
2.69k stars 389 forks source link

[Bug]: Cannot locate py3d_tools module when started with --nowebui parameter #830

Closed nberktumer closed 1 year ago

nberktumer commented 1 year ago

Have you read the latest version of the FAQ?

Is there an existing issue for this?

Are you using the latest version of the Deforum extension?

What happened?

I tried running deforum using the API endpoints with no webui. However, when I send a job to deforum_api/batches endpoint, I get the following exception:

Traceback (most recent call last):
  File "/home/ubuntu/app/extensions/deforum/scripts/deforum_api.py", line 81, in run_deforum_batch
    run_deforum(*run_deforum_args)
  File "/home/ubuntu/app/extensions/deforum/scripts/deforum_helpers/run_deforum.py", line 82, in run_deforum
    from .render import render_animation
  File "/home/ubuntu/app/extensions/deforum/scripts/deforum_helpers/render.py", line 12, in <module>
    from .noise import add_noise
  File "/home/ubuntu/app/extensions/deforum/scripts/deforum_helpers/noise.py", line 6, in <module>
    from .animation import sample_to_cv2
  File "/home/ubuntu/app/extensions/deforum/scripts/deforum_helpers/animation.py", line 3, in <module>
    import py3d_tools as p3d # this is actually a file in our /src folder!
ModuleNotFoundError: No module named 'py3d_tools'

This exception only occurs when I start the webui launcher with--nowebui parameter.

Steps to reproduce the problem

  1. Start webui with --nowebui parameter:
    • python launch.py --no-prepare --api --deforum-api --nowebui --listen --no-hashing --device-id 0 --port 7860 --opt-sdp-no-mem-attention
  2. Start any deforum job by doing a POST request to deforum_api/batches endpoint.

What should have happened/how would you fix it?

deforum_sys_extend() function is only called for the WebUI process and it is not called for the Deforum API processes. Hence, API processes cannot locate py3d_tools modules.

This function should also have been called during FastAPI initialization.

Torch version

2.0.1

On which platform are you launching the webui with the extension?

Local PC setup (Linux), Cloud server (Linux)

Deforum settings

{ "W": 512, "H": 512, "show_info_on_ui": true, "tiling": false, "restore_faces": false, "seed_resize_from_w": 0, "seed_resize_from_h": 0, "seed": 1, "sampler": "Euler a", "steps": 1, "batchname": "Deforum{timestring}", "seed_behavior": "iter", "seed_iter_N": 1, "use_init": false, "strength": 0.8, "strength_0_no_init": true, "init_image": "None", "use_mask": false, "use_alpha_as_mask": false, "mask_file": "", "invert_mask": false, "mask_contrast_adjust": 1.0, "mask_brightness_adjust": 1.0, "overlay_mask": true, "mask_overlay_blur": 4, "fill": 1, "full_res_mask": true, "full_res_mask_padding": 4, "reroll_blank_frames": "ignore", "reroll_patience": 10.0, "prompts": { "0": "tiny cute swamp bunny, highly detailed, intricate, ultra hd, sharp photo, crepuscular rays, in focus, by tomasz alen kopera", "30": "anthropomorphic clean cat, surrounded by fractals, epic angle and pose, symmetrical, 3d, depth of field, ruan jia and fenghua zhong", "60": "a beautiful coconut --neg photo, realistic", "90": "a beautiful durian, trending on Artstation" }, "animation_prompts_positive": "", "animation_prompts_negative": "", "animation_mode": "2D", "max_frames": 5, "border": "replicate", "angle": "0:(0)", "zoom": "0:(1.0025+0.002sin(1.253.14t/30))", "translation_x": "0:(0)", "translation_y": "0:(0)", "translation_z": "0:(0)", "transform_center_x": "0:(0.5)", "transform_center_y": "0:(0.5)", "rotation_3d_x": "0:(0)", "rotation_3d_y": "0:(0)", "rotation_3d_z": "0:(0)", "enable_perspective_flip": false, "perspective_flip_theta": "0:(0)", "perspective_flip_phi": "0:(0)", "perspective_flip_gamma": "0:(0)", "perspective_flip_fv": "0:(53)", "noise_schedule": "0: (0.04)", "strength_schedule": "0: (0.65)", "contrast_schedule": "0: (1.0)", "cfg_scale_schedule": "0: (7)", "enable_steps_scheduling": false, "steps_schedule": "0: (25)", "fov_schedule": "0: (70)", "aspect_ratio_schedule": "0: (1)", "aspect_ratio_use_old_formula": false, "near_schedule": "0: (200)", "far_schedule": "0: (10000)", "seed_schedule": "0:(s), 1:(-1), \"max_f-2\":(-1), \"max_f-1\":(s)", "pix2pix_img_cfg_scale_schedule": "0:(1.5)", "enable_subseed_scheduling": false, "subseed_schedule": "0:(1)", "subseed_strength_schedule": "0:(0)", "enable_sampler_scheduling": false, "sampler_schedule": "0: (\"Euler a\")", "use_noise_mask": false, "mask_schedule": "0: (\"{video_mask}\")", "noise_mask_schedule": "0: (\"{video_mask}\")", "enable_checkpoint_scheduling": false, "checkpoint_schedule": "0: (\"model1.ckpt\"), 100: (\"model2.safetensors\")", "enable_clipskip_scheduling": false, "clipskip_schedule": "0: (2)", "enable_noise_multiplier_scheduling": false, "noise_multiplier_schedule": "0: (1.05)", "resume_from_timestring": false, "resume_timestring": "20230707221541", "enable_ddim_eta_scheduling": false, "ddim_eta_schedule": "0: (0)", "enable_ancestral_eta_scheduling": false, "ancestral_eta_schedule": "0: (1)", "amount_schedule": "0: (0.05)", "kernel_schedule": "0: (5)", "sigma_schedule": "0: (1.0)", "threshold_schedule": "0: (0.0)", "color_coherence": "LAB", "color_coherence_image_path": "", "color_coherence_video_every_N_frames": 1, "color_force_grayscale": false, "legacy_colormatch": false, "diffusion_cadence": 1, "optical_flow_cadence": "None", "cadence_flow_factor_schedule": "0: (1)", "optical_flow_redo_generation": "None", "redo_flow_factor_schedule": "0: (1)", "diffusion_redo": 0, "noise_type": "perlin", "perlin_octaves": 4, "perlin_persistence": 0.5, "use_depth_warping": true, "depth_algorithm": "Zoe", "midas_weight": 0.4, "padding_mode": "reflection", "sampling_mode": "bicubic", "save_depth_maps": false, "video_init_path": "", "extract_nth_frame": 1, "extract_from_frame": 0, "extract_to_frame": -1, "overwrite_extracted_frames": false, "use_mask_video": false, "video_mask_path": "", "hybrid_comp_alpha_schedule": "0:(0.5)", "hybrid_comp_mask_blend_alpha_schedule": "0:(0.5)", "hybrid_comp_mask_contrast_schedule": "0:(1)", "hybrid_comp_mask_auto_contrast_cutoff_high_schedule": "0:(100)", "hybrid_comp_mask_auto_contrast_cutoff_low_schedule": "0:(0)", "hybrid_flow_factor_schedule": "0:(1)", "hybrid_generate_inputframes": false, "hybrid_generate_human_masks": "None", "hybrid_use_first_frame_as_init_image": true, "hybrid_motion": "None", "hybrid_motion_use_prev_img": false, "hybrid_flow_consistency": false, "hybrid_consistency_blur": 2, "hybrid_flow_method": "RAFT", "hybrid_composite": "None", "hybrid_use_init_image": false, "hybrid_comp_mask_type": "None", "hybrid_comp_mask_inverse": false, "hybrid_comp_mask_equalize": "None", "hybrid_comp_mask_auto_contrast": true, "hybrid_comp_save_extra_frames": false, "parseq_manifest": "", "parseq_use_deltas": true, "use_looper": false, "init_images": "{\n \"0\": \"https://deforum.github.io/a1/Gi1.png\",\n \"max_f/4-5\": \"https://deforum.github.io/a1/Gi2.png\",\n \"max_f/2-10\": \"https://deforum.github.io/a1/Gi3.png\",\n \"3max_f/4-15\": \"https://deforum.github.io/a1/Gi4.jpg\",\n \"max_f-20\": \"https://deforum.github.io/a1/Gi1.png\"\n}", "image_strength_schedule": "0:(0.75)", "blendFactorMax": "0:(0.35)", "blendFactorSlope": "0:(0.25)", "tweening_frames_schedule": "0:(20)", "color_correction_factor": "0:(0.075)", "cn_1_overwrite_frames": true, "cn_1_vid_path": "", "cn_1_mask_vid_path": "", "cn_1_enabled": false, "cn_1_low_vram": false, "cn_1_pixel_perfect": false, "cn_1_module": "none", "cn_1_model": "control_v11f1p_sd15_depth [cfd03158]", "cn_1_weight": "0:(1)", "cn_1_guidance_start": "0:(0.0)", "cn_1_guidance_end": "0:(1.0)", "cn_1_processor_res": 64, "cn_1_threshold_a": 64, "cn_1_threshold_b": 64, "cn_1_resize_mode": "Inner Fit (Scale to Fit)", "cn_1_control_mode": "Balanced", "cn_1_loopback_mode": false, "cn_2_overwrite_frames": true, "cn_2_vid_path": "", "cn_2_mask_vid_path": "", "cn_2_enabled": false, "cn_2_low_vram": false, "cn_2_pixel_perfect": false, "cn_2_module": "none", "cn_2_model": "control_v11p_sd15_seg [e1f51eb9]", "cn_2_weight": "0:(1)", "cn_2_guidance_start": "0:(0.0)", "cn_2_guidance_end": "0:(1.0)", "cn_2_processor_res": 64, "cn_2_threshold_a": 64, "cn_2_threshold_b": 64, "cn_2_resize_mode": "Inner Fit (Scale to Fit)", "cn_2_control_mode": "Balanced", "cn_2_loopback_mode": false, "cn_3_overwrite_frames": true, "cn_3_vid_path": "", "cn_3_mask_vid_path": "", "cn_3_enabled": false, "cn_3_low_vram": false, "cn_3_pixel_perfect": false, "cn_3_module": "none", "cn_3_model": "None", "cn_3_weight": "0:(1)", "cn_3_guidance_start": "0:(0.0)", "cn_3_guidance_end": "0:(1.0)", "cn_3_processor_res": 64, "cn_3_threshold_a": 64, "cn_3_threshold_b": 64, "cn_3_resize_mode": "Inner Fit (Scale to Fit)", "cn_3_control_mode": "Balanced", "cn_3_loopback_mode": false, "cn_4_overwrite_frames": true, "cn_4_vid_path": "", "cn_4_mask_vid_path": "", "cn_4_enabled": false, "cn_4_low_vram": false, "cn_4_pixel_perfect": false, "cn_4_module": "none", "cn_4_model": "None", "cn_4_weight": "0:(1)", "cn_4_guidance_start": "0:(0.0)", "cn_4_guidance_end": "0:(1.0)", "cn_4_processor_res": 64, "cn_4_threshold_a": 64, "cn_4_threshold_b": 64, "cn_4_resize_mode": "Inner Fit (Scale to Fit)", "cn_4_control_mode": "Balanced", "cn_4_loopback_mode": false, "cn_5_overwrite_frames": true, "cn_5_vid_path": "", "cn_5_mask_vid_path": "", "cn_5_enabled": false, "cn_5_low_vram": false, "cn_5_pixel_perfect": false, "cn_5_module": "none", "cn_5_model": "None", "cn_5_weight": "0:(1)", "cn_5_guidance_start": "0:(0.0)", "cn_5_guidance_end": "0:(1.0)", "cn_5_processor_res": 64, "cn_5_threshold_a": 64, "cn_5_threshold_b": 64, "cn_5_resize_mode": "Inner Fit (Scale to Fit)", "cn_5_control_mode": "Balanced", "cn_5_loopback_mode": false, "skip_video_creation": false, "fps": 20, "make_gif": false, "delete_imgs": false, "delete_input_frames": false, "add_soundtrack": "None", "soundtrack_path": "", "r_upscale_video": false, "r_upscale_factor": "x4", "r_upscale_model": "realesrgan-x4plus", "r_upscale_keep_imgs": true, "store_frames_in_ram": false, "frame_interpolation_engine": "None", "frame_interpolation_x_amount": 3, "frame_interpolation_slow_mo_enabled": false, "frame_interpolation_slow_mo_amount": 2, "frame_interpolation_keep_imgs": false, "frame_interpolation_use_upscaled": false }

Webui core settings

[]

Console logs

Launching API server with arguments: --no-prepare --api --deforum-api --nowebui --listen --no-hashing --device-id 0 --port 7860 --opt-sdp-no-mem-attention --model-lru-cache-size 20
no module 'xformers'. Processing without...
no module 'xformers'. Processing without...
No module 'xformers'. Proceeding without it.
2023-08-08 09:24:21,953 - ControlNet - INFO - ControlNet v1.1.234
ControlNet preprocessor location: /home/ubuntu/app/extensions/sd-webui-controlnet/annotator/downloads
2023-08-08 09:24:22,172 - ControlNet - INFO - ControlNet v1.1.234
Loading weights [None] from /home/ubuntu/app/models/Stable-diffusion/SD_1.5.ckpt
Startup time: 17.6s (start: 0.3s, launcher: 4.7s, import torch: 4.7s, setup paths: 1.5s, other imports: 4.3s, setup codeformer: 0.1s, load scripts: 1.8s, launch.py: 0.1s).
INFO:     Started server process [5556]
INFO:     Waiting for application startup.
INFO:     Application startup complete.
INFO:     Uvicorn running on http://0.0.0.0:7860 (Press CTRL+C to quit)
Creating model from config: /home/ubuntu/app/configs/v1-inference.yaml
LatentDiffusion: Running in eps-prediction mode
DiffusionWrapper has 859.52 M params.
Downloading (…)olve/main/vocab.json: 100%|███| 961k/961k [00:00<00:00, 4.48MB/s]
Downloading (…)olve/main/merges.txt: 100%|███| 525k/525k [00:00<00:00, 3.69MB/s]
Downloading (…)cial_tokens_map.json: 100%|█████| 389/389 [00:00<00:00, 2.53MB/s]
Downloading (…)okenizer_config.json: 100%|█████| 905/905 [00:00<00:00, 4.33MB/s]
Downloading (…)lve/main/config.json: 100%|█| 4.52k/4.52k [00:00<00:00, 24.9MB/s]
Applying attention optimization: sdp-no-mem... done.
Model loaded in 5.1s (load weights from disk: 1.3s, create model: 2.0s, apply weights to model: 1.0s, apply half(): 0.3s, move model to device: 0.3s).
INFO:     188.119.40.33:24561 - "GET /docs HTTP/1.1" 200 OK
INFO:     188.119.40.33:24561 - "GET /openapi.json HTTP/1.1" 200 OK
2023-08-08 09:27:46 INFO [deforum_api] Starting batch batch(162391475) in thread 139991611074112.
INFO:     188.119.40.33:24561 - "POST /deforum_api/batches HTTP/1.1" 202 Accepted
*Deforum ControlNet support: enabled*
started run_deforum
times_to_run: 1
Deforum extension for auto1111 webui
Git commit: ff0680bb
Starting job batch(162391475)-0...
reading custom settings from /tmp/tmpx6shdjvv
namespace(W=512, H=512, show_info_on_ui=True, tiling=False, restore_faces=False, seed_resize_from_w=0, seed_resize_from_h=0, seed=1, sampler='Euler a', steps=1, batch_name='Deforum_{timestring}', seed_behavior='iter', seed_iter_N=1, use_init=False, strength=0.8, strength_0_no_init=True, init_image='None', use_mask=False, use_alpha_as_mask=False, mask_file='', invert_mask=False, mask_contrast_adjust=1.0, mask_brightness_adjust=1.0, overlay_mask=True, mask_overlay_blur=4, fill=1, full_res_mask=True, full_res_mask_padding=4, reroll_blank_frames='ignore', reroll_patience=10.0) namespace(animation_mode='2D', max_frames=5, border='replicate', angle='0:(0)', zoom='0:(1.0025+0.002*sin(1.25*3.14*t/30))', translation_x='0:(0)', translation_y='0:(0)', translation_z='0:(0)', transform_center_x='0:(0.5)', transform_center_y='0:(0.5)', rotation_3d_x='0:(0)', rotation_3d_y='0:(0)', rotation_3d_z='0:(0)', enable_perspective_flip=False, perspective_flip_theta='0:(0)', perspective_flip_phi='0:(0)', perspective_flip_gamma='0:(0)', perspective_flip_fv='0:(53)', noise_schedule='0: (0.04)', strength_schedule='0: (0.65)', contrast_schedule='0: (1.0)', cfg_scale_schedule='0: (7)', enable_steps_scheduling=False, steps_schedule='0: (25)', fov_schedule='0: (70)', aspect_ratio_schedule='0: (1)', aspect_ratio_use_old_formula=False, near_schedule='0: (200)', far_schedule='0: (10000)', seed_schedule='0:(s), 1:(-1), "max_f-2":(-1), "max_f-1":(s)', pix2pix_img_cfg_scale_schedule='0:(1.5)', enable_subseed_scheduling=False, subseed_schedule='0:(1)', subseed_strength_schedule='0:(0)', enable_sampler_scheduling=False, sampler_schedule='0: ("Euler a")', use_noise_mask=False, mask_schedule='0: ("{video_mask}")', noise_mask_schedule='0: ("{video_mask}")', enable_checkpoint_scheduling=False, checkpoint_schedule='0: ("model1.ckpt"), 100: ("model2.safetensors")', enable_clipskip_scheduling=False, clipskip_schedule='0: (2)', enable_noise_multiplier_scheduling=False, noise_multiplier_schedule='0: (1.05)', resume_from_timestring=False, resume_timestring='20230707221541', enable_ddim_eta_scheduling=False, ddim_eta_schedule='0: (0)', enable_ancestral_eta_scheduling=False, ancestral_eta_schedule='0: (1)', amount_schedule='0: (0.05)', kernel_schedule='0: (5)', sigma_schedule='0: (1.0)', threshold_schedule='0: (0.0)', color_coherence='LAB', color_coherence_image_path='', color_coherence_video_every_N_frames=1, color_force_grayscale=False, legacy_colormatch=False, diffusion_cadence=1, optical_flow_cadence='None', cadence_flow_factor_schedule='0: (1)', optical_flow_redo_generation='None', redo_flow_factor_schedule='0: (1)', diffusion_redo=0, noise_type='perlin', perlin_w=8, perlin_h=8, perlin_octaves=4, perlin_persistence=0.5, use_depth_warping=True, depth_algorithm='Zoe', midas_weight=0.4, padding_mode='reflection', sampling_mode='bicubic', save_depth_maps=False, video_init_path='', extract_nth_frame=1, extract_from_frame=0, extract_to_frame=-1, overwrite_extracted_frames=False, use_mask_video=False, video_mask_path='', hybrid_comp_alpha_schedule='0:(0.5)', hybrid_comp_mask_blend_alpha_schedule='0:(0.5)', hybrid_comp_mask_contrast_schedule='0:(1)', hybrid_comp_mask_auto_contrast_cutoff_high_schedule='0:(100)', hybrid_comp_mask_auto_contrast_cutoff_low_schedule='0:(0)', hybrid_flow_factor_schedule='0:(1)', hybrid_generate_inputframes=False, hybrid_generate_human_masks='None', hybrid_use_first_frame_as_init_image=True, hybrid_motion='None', hybrid_motion_use_prev_img=False, hybrid_flow_consistency=False, hybrid_consistency_blur=2, hybrid_flow_method='RAFT', hybrid_composite='None', hybrid_use_init_image=False, hybrid_comp_mask_type='None', hybrid_comp_mask_inverse=False, hybrid_comp_mask_equalize='None', hybrid_comp_mask_auto_contrast=True, hybrid_comp_save_extra_frames=False) namespace(parseq_manifest='', parseq_use_deltas=True) namespace(use_looper=False, init_images='{\n    "0": "https://deforum.github.io/a1/Gi1.png",\n    "max_f/4-5": "https://deforum.github.io/a1/Gi2.png",\n    "max_f/2-10": "https://deforum.github.io/a1/Gi3.png",\n    "3*max_f/4-15": "https://deforum.github.io/a1/Gi4.jpg",\n    "max_f-20": "https://deforum.github.io/a1/Gi1.png"\n}', image_strength_schedule='0:(0.75)', blendFactorMax='0:(0.35)', blendFactorSlope='0:(0.25)', tweening_frames_schedule='0:(20)', color_correction_factor='0:(0.075)')
2023-08-08 09:27:47 WARNING [deforum_api] Error during batch execution: <class 'ModuleNotFoundError'> - No module named 'py3d_tools'
2023-08-08 09:27:47 ERROR [deforum_api] Batch batch(162391475) failed: No module named 'py3d_tools'
Traceback (most recent call last):
  File "/home/ubuntu/app/extensions/deforum/scripts/deforum_api.py", line 81, in run_deforum_batch
    run_deforum(*run_deforum_args)
  File "/home/ubuntu/app/extensions/deforum/scripts/deforum_helpers/run_deforum.py", line 82, in run_deforum
    from .render import render_animation
  File "/home/ubuntu/app/extensions/deforum/scripts/deforum_helpers/render.py", line 12, in <module>
    from .noise import add_noise
  File "/home/ubuntu/app/extensions/deforum/scripts/deforum_helpers/noise.py", line 6, in <module>
    from .animation import sample_to_cv2
  File "/home/ubuntu/app/extensions/deforum/scripts/deforum_helpers/animation.py", line 3, in <module>
    import py3d_tools as p3d # this is actually a file in our /src folder!
ModuleNotFoundError: No module named 'py3d_tools'

Additional information

No response

github-actions[bot] commented 1 year ago

This issue has been closed due to incorrect formatting. Please address the following mistakes and reopen the issue (click on the 'Reopen' button below):