Version of xformers

[Trainingzy]

Thanks for your great work.

I cannot run the script run_videop2p.py on either 4090 or A5000. I have tried three versions of xformers.

xformers 0.0.15.dev0+0bad001.d20230429 will lead to this error:

NotImplementedError: No operator found for this attention: AttentionOpDispatch(dtype=torch.float32, device=device(type='cuda', index=0), k=80, has_dropout=False, attn_bias_type=<class 'NoneType'>, kv_len=1024, q_len=1024, kv=80, batch_size=64, num_heads=1, has_custom_scale=False, requires_grad=True)

xformers 0.0.16 and xformers 0.0.17 will lead to out-of-memoryerror.

Also, I have tried both pytorch 1.12.1 and 1.13.1. Neither of them work.

May I know the xformers version you use?

[ShaoTengLiu]

My xformers is 0.0.15.dev+0bad001.d20230409. It can work on both V100 and 3090.

It is known that xformers may have different bugs on different devices. I don't have 4090 or A5000 to debug.

Here are some related issues I found: issue1, issue2. Hope it can help you.

[Trainingzy]

Thanks for the reply. The model can work with xformers 0.0.17 in my situation now.

The problem is the VRAM. On 4090/A5000, I can only run the --fast version, which requires about 18GB. On V100, the normal version can be used, taking up about 28GB.

What is the difference between the normal and fast version?

[ShaoTengLiu]

You can refer to this line. The fast mode doesn't use classifier-free guidance.

[ShaoTengLiu]

I will temporally close this issue. You are welcome to reopen it if you still have this problem.