problem with triton

[gonduras]

(venv) E:\ControlVideo>python inference.py A matching Triton is not available, some optimizations will not be enabled. Error caught was: No module named 'triton.language' Traceback (most recent call last): File "E:\ControlVideo\inference.py", line 20, in from models.pipeline_controlvideo import ControlVideoPipeline File "E:\ControlVideo\models\pipeline_controlvideo.py", line 28, in from .controlnet import ControlNetOutput File "E:\ControlVideo\models\controlnet.py", line 27, in from .controlnet_unet_blocks import ( File "E:\ControlVideo\models\controlnet_unet_blocks.py", line 6, in from .controlnet_attention import Transformer3DModel File "E:\ControlVideo\models\controlnet_attention.py", line 15, in from diffusers.models.attention import CrossAttention, FeedForward, AdaLayerNorm ImportError: cannot import name 'CrossAttention' from 'diffusers.models.attention' (E:\ControlVideo\venv\lib\site-packages\diffusers\models\attention.py)

[JPW0080]

pip install https://huggingface.co/r4ziel/xformers_pre_built/resolve/main/triton-2.0.0-cp310-cp310-win_amd64.whl

[gonduras]

thank you! can you help me with the setup? now i get this:

(venv) E:\control-a-video>python inference.py --prompt "gondurastration" --input_video bear.mp4 --control_mode depth A matching Triton is not available, some optimizations will not be enabled. Error caught was: DLL load failed while importing libtriton: The specified module could not be found. controlnet\diffusion_pytorch_model.safetensors not found

i put in the directory controlnet files from here https://huggingface.co/lllyasviel/sd-controlnet-depth/blob/main/diffusion_pytorch_model.safetensors

[JPW0080]

git pull for latest update pip install -r requirements.txt

here is my folder structure,

checkpoints folder contains:

models--lllyasviel--ControlNet sd-controlnet-canny sd-controlnet-depth sd-controlnet-openpose stable-diffusion-v1-5 flownet.pkl

And here is a command that was used for 1.0 ControlNet,

python inference.py --prompt "Solid platinum, Statue of Liberty, moonwalk" --condition depth_midas --video_path "data/moonwalk.mp4" --output_path "outputs" --video_length 15 --smoother_steps 19 20 --width 512 --height 512 --frame_rate 2 --version v10

Lastly, https://github.com/YBYBZhang/ControlVideo/blob/master/inference.py

Take a look at the def get_args(): Section

[gonduras]

thanks @JPW0080 still struggling

(venv) E:\ControlVideo>python inference.py --prompt "gondurastration" --condition depth_midas --video_path "data/moonwalk.mp4" --output_path "outputs" --video_length 15 --smoother_steps 19 20 --width 512 --height 512 --frame_rate 2 --version v10 Downloading (…)id-midas-501f0c75.pt: 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 493M/493M [07:07<00:00, 1.15MB/s] E:\ControlVideo\venv\lib\site-packages\timm\models_factory.py:114: UserWarning: Mapping deprecated model name vit_base_resnet50_384 to current vit_base_r50_s16_384.orig_in21k_ft_in1k. model = create_fn( Traceback (most recent call last): File "E:\ControlVideo\inference.py", line 98, in vae = AutoencoderKL.from_pretrained(sd_path, subfolder="vae").to(dtype=torch.float16) File "E:\ControlVideo\venv\lib\site-packages\diffusers\models\modeling_utils.py", line 558, in from_pretrained model = cls.from_config(config, unused_kwargs) File "E:\ControlVideo\venv\lib\site-packages\diffusers\configuration_utils.py", line 210, in from_config model = cls(init_dict) File "E:\ControlVideo\venv\lib\site-packages\diffusers\configuration_utils.py", line 569, in inner_init init(self, *args, **init_kwargs) File "E:\ControlVideo\venv\lib\site-packages\diffusers\models\autoencoder_kl.py", line 86, in init self.encoder = Encoder( File "E:\ControlVideo\venv\lib\site-packages\diffusers\models\vae.py", line 65, in init down_block = get_down_block( File "E:\ControlVideo\venv\lib\site-packages\diffusers\models\unet_2d_blocks.py", line 88, in get_down_block raise ValueError("cross_attention_dim must be specified for CrossAttnDownBlock2D") ValueError: cross_attention_dim must be specified for CrossAttnDownBlock2D

[JPW0080]

After revisiting ControlVideo, here was the full Installation Procedure

Install Anaconda3 and Launch Anaconda3 Prompt

conda create -n controlvideo python=3.10.6 conda activate controlvideo mkdir \AI cd \AI git clone https://github.com/YBYBZhang/ControlVideo.git mkdir \AI\ControlVideo\output cd \AI\ControlVideo pip install https://download.pytorch.org/whl/cu116/torchvision-0.14.1%2Bcu116-cp310-cp310-win_amd64.whl pip install https://download.pytorch.org/whl/cu116/torch-1.13.1%2Bcu116-cp310-cp310-win_amd64.whl pip install git+https://github.com/openai/CLIP.git pip install matplotlib pip install mediapipe pip install positional-encodings

Now, minimize Anaconda3 Prompt

Locate the requirements.txt in the \AI\ControlVideo directory Delete these lines and save the file

clip==1.0 deepspeed==0.8.0 torch==1.13.1+cu116 torchvision==0.14.1+cu116

Refer to https://github.com/YBYBZhang/ControlVideo and download the needed "diffusers" weights as described.

Maximize Anaconda3 Prompt

pip install -r requirements.txt

python inference.py --prompt "Solid platinum, Statue of Liberty, moonwalk" --video_path ./data/moonwalk.mp4 --output_path ./output/ --condition depth_midas --video_length 15 --width 512 --height 512 --frame_rate 2 --smoother_steps 19 20 --version v10 --seed 2023

Disregard Triton Error, if all goes well, Inference should commence.

[rB080]

Hi, I didn't exactly understand how to download the weights and getting the missing weight file error as expected. Can I wget the weights some way?