some error happened..

[alanhuang67]

Hi,

it not always but there are some error happened when generating , error as below, any idea about this? thanks!

     INFO     len( region_list )=3                                                                  generate.py:1154

30% ━━━━━━━━━━━━━━━━━━━━━━╸━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 8/27 [ 0:01:05 < 0:02:18 , 0 it/s ] ╭─────────────────────────────── Traceback (most recent call last) ────────────────────────────────╮ │ D:\animatediff-cli-prompt-travel\src\animatediff\cli.py:392 in generate │ │ │ │ 389 │ │ │ logger.info(f"Generation seed: {seed}") │ │ 390 │ │ │ │ │ 391 │ │ │ │ │ ❱ 392 │ │ │ output = run_inference( │ │ 393 │ │ │ │ pipeline=g_pipeline, │ │ 394 │ │ │ │ n_prompt=n_prompt, │ │ 395 │ │ │ │ seed=seed, │ │ │ │ D:\animatediff-cli-prompt-travel\src\animatediff\generate.py:1156 in run_inference │ │ │ │ 1153 │ logger.info(f"{len( region_condi_list )=}") │ │ 1154 │ logger.info(f"{len( region_list )=}") │ │ 1155 │ │ │ ❱ 1156 │ pipeline_output = pipeline( │ │ 1157 │ │ negative_prompt=n_prompt, │ │ 1158 │ │ num_inference_steps=steps, │ │ 1159 │ │ guidance_scale=guidance_scale, │ │ │ │ D:\animatediff-cli-prompt-travel\venv\lib\site-packages\torch\utils_contextlib.py:115 in │ │ decorate_context │ │ │ │ 112 │ @functools.wraps(func) │ │ 113 │ def decorate_context(*args, *kwargs): │ │ 114 │ │ with ctx_factory(): │ │ ❱ 115 │ │ │ return func(args, **kwargs) │ │ 116 │ │ │ 117 │ return decorate_context │ │ 118 │ │ │ │ D:\animatediff-cli-prompt-travel\src\animatediff\pipelines\animation.py:2849 in call │ │ │ │ 2846 │ │ │ │ │ │ controlnet_target = [f%video_length for f in controlnet_target] │ │ 2847 │ │ │ │ │ │ controlnet_target = list(set(controlnet_target)) │ │ 2848 │ │ │ │ │ │ │ │ ❱ 2849 │ │ │ │ │ │ process_controlnet(controlnet_target) │ │ 2850 │ │ │ │ │ │ │ 2851 │ │ │ │ │ # expand the latents if we are doing classifier free guidance │ │ 2852 │ │ │ │ │ latent_model_input = ( │ │ │ │ D:\animatediff-cli-prompt-travel\src\animatediff\pipelines\animation.py:2774 in │ │ process_controlnet │ │ │ │ 2771 │ │ │ │ │ │ │ frame_no = cont_var["frame_no"] │ │ 2772 │ │ │ │ │ │ │ │ │ 2773 │ │ │ │ │ │ │ latent_model_input = ( │ │ ❱ 2774 │ │ │ │ │ │ │ │ latents[:, :, [frame_no]] │ │ 2775 │ │ │ │ │ │ │ │ .to(device) │ │ 2776 │ │ │ │ │ │ │ │ .repeat( prompt_encoder.get_condi_size(), 1, 1, 1, 1) │ │ 2777 │ │ │ │ │ │ │ ) │ ╰──────────────────────────────────────────────────────────────────────────────────────────────────╯ RuntimeError: CUDA error: unknown error CUDA kernel errors might be asynchronously reported at some other API call, so the stacktrace below might be incorrect. For debugging consider passing CUDA_LAUNCH_BLOCKING=1. Compile with TORCH_USE_CUDA_DSA to enable device-side assertions.

[aduchon]

try a bigger GPU

[alanhuang67]

sounds sad...already 4080 with 12G vram....

[alanhuang67]

I'm trying to look for the solution, not sure if this is related....

https://github.com/pytorch/pytorch/issues/60600

CUDA unknown error - this may be due to an incorrectly set up environment, e.g. changing env variable CUDA_VISIBLE_DEVICES after program start. Setting the available devices to be zero. #60600

@s9roll7

[aduchon]

if you're on a Google Colab, try:

!pip install -q torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu118
!pip install -q tensorrt
!pip install -U xformers --index-url https://download.pytorch.org/whl/cu118
!pip install -q  imageio

!pip install -q controlnet_aux
!pip install -q transformers==4.34.1
!pip install -q mediapipe onnxruntime
!pip install -q omegaconf

!pip install -q ffmpeg-python

# have to use 0.18.1 to avoid error: ImportError: cannot import name 'maybe_allow_in_graph' from 'diffusers.utils' (/usr/local/lib/python3.10/dist-packages/diffusers/utils/__init__.py)
!pip install -q diffusers[torch]==0.18.1

%set_env PYTHONPATH=/content/drive/MyDrive/AI/AnimateDiff/animatediff-cli-prompt-travel/src

[alanhuang67]

not familiar with colab...

[alanhuang67]

I find out there is a message about CUDA, is this the reason for why I have this issue?

D:\animatediff-cli-prompt-travel\venv\lib\site-packages\onnxruntime\capi\onnxruntime_inference_collection.py:69: UserWarning: Specified provider 'CUDAExecutionProvider' is not in available provider names.Available providers: 'AzureExecutionProvider, CPUExecutionProvider' warnings.warn( Preprocessing images (controlnet_openpose) 22% ━━━━━━━╺━━━━━━━━━━━━━━━━━━━━━━━━━ 74/329 [ 0:00:27 < 0:01:33 , 3 it/s ]

@aduchon @s9roll7