[Bug]: TypeError: bad operand type for unary ~: 'NoneType'

[tin2tin]

Is there an existing issue for this?

• [X] I have searched the existing issues and checked the recent builds/commits of both this extension and the webui

Are you using the latest version of the extension?

• [X] I have the modelscope text2video extension updated to the lastest version and I still have the issue.

What happened?

Error after the latest update.

Steps to reproduce the problem

1. Enter the settings
2. Render

What should have happened?

No error

WebUI and Deforum extension Commit IDs

webui commit id - txt2vid commit id - 3acf729d (Sun Apr 16 11:24:15 2023)

What GPU were you using for launching?

RTX 2060 6GB

On which platform are you launching the webui backend with the extension?

No response

Settings

Console logs

Error report:

Git commit: 3acf729d (Sun Apr 16 11:24:15 2023)
Starting text2video
Pipeline setup
config namespace(framework='pytorch', task='text-to-video-synthesis', model={'type': 'latent-text-to-video-synthesis', 'model_args': {'ckpt_clip': 'open_clip_pytorch_model.bin', 'ckpt_unet': 'text2video_pytorch_model.pth', 'ckpt_autoencoder': 'VQGAN_autoencoder.pth', 'max_frames': 16, 'tiny_gpu': 1}, 'model_cfg': {'unet_in_dim': 4, 'unet_dim': 320, 'unet_y_dim': 768, 'unet_context_dim': 1024, 'unet_out_dim': 4, 'unet_dim_mult': [1, 2, 4, 4], 'unet_num_heads': 8, 'unet_head_dim': 64, 'unet_res_blocks': 2, 'unet_attn_scales': [1, 0.5, 0.25], 'unet_dropout': 0.1, 'temporal_attention': 'True', 'num_timesteps': 1000, 'mean_type': 'eps', 'var_type': 'fixed_small', 'loss_type': 'mse'}}, pipeline={'type': 'latent-text-to-video-synthesis'})
device cuda
Working in txt2vid mode
  0%|                                                                                            | 0/1 [00:00<?, ?it/s]latents torch.Size([1, 4, 18, 32, 64]) tensor(0.0020, device='cuda:0') tensor(0.9992, device='cuda:0')
DDIM sampling:   0%|                                                                            | 0/31 [00:00<?, ?it/s]
Traceback (most recent call last):                                                              | 0/31 [00:00<?, ?it/s]
  File "C:\Users\45239\Documents\GitHub\stable-diffusion-webui/extensions/sd-webui-modelscope-text2video/scripts\t2v_helpers\render.py", line 24, in run
    vids_pack = process_modelscope(args_dict)
  File "C:\Users\45239\Documents\GitHub\stable-diffusion-webui/extensions/sd-webui-modelscope-text2video/scripts\modelscope\process_modelscope.py", line 193, in process_modelscope
    samples, _ = pipe.infer(args.prompt, args.n_prompt, args.steps, args.frames, args.seed + batch if args.seed != -1 else -1, args.cfg_scale,
  File "C:\Users\45239\Documents\GitHub\stable-diffusion-webui/extensions/sd-webui-modelscope-text2video/scripts\modelscope\t2v_pipeline.py", line 245, in infer
    x0 = self.diffusion.ddim_sample_loop(
  File "C:\Users\45239\Documents\GitHub\stable-diffusion-webui\venv\lib\site-packages\torch\autograd\grad_mode.py", line 27, in decorate_context
    return func(*args, **kwargs)
  File "C:\Users\45239\Documents\GitHub\stable-diffusion-webui\extensions\sd-webui-modelscope-text2video\scripts\modelscope\t2v_model.py", line 1469, in ddim_sample_loop
    xt = self.ddim_sample(xt, t, model, model_kwargs, clamp,
  File "C:\Users\45239\Documents\GitHub\stable-diffusion-webui\venv\lib\site-packages\torch\autograd\grad_mode.py", line 27, in decorate_context
    return func(*args, **kwargs)
  File "C:\Users\45239\Documents\GitHub\stable-diffusion-webui\extensions\sd-webui-modelscope-text2video\scripts\modelscope\t2v_model.py", line 1321, in ddim_sample
    _, _, _, x0 = self.p_mean_variance(xt, t, model, model_kwargs, clamp,
  File "C:\Users\45239\Documents\GitHub\stable-diffusion-webui\extensions\sd-webui-modelscope-text2video\scripts\modelscope\t2v_model.py", line 1262, in p_mean_variance
    y_out = model(xt, self._scale_timesteps(t), **model_kwargs[0])
  File "C:\Users\45239\Documents\GitHub\stable-diffusion-webui\venv\lib\site-packages\torch\nn\modules\module.py", line 1194, in _call_impl
    return forward_call(*input, **kwargs)
  File "C:\Users\45239\Documents\GitHub\stable-diffusion-webui\extensions\sd-webui-modelscope-text2video\scripts\modelscope\t2v_model.py", line 367, in forward
    x = self._forward_single(block, x, e, context, time_rel_pos_bias,
  File "C:\Users\45239\Documents\GitHub\stable-diffusion-webui\extensions\sd-webui-modelscope-text2video\scripts\modelscope\t2v_model.py", line 430, in _forward_single
    x = self._forward_single(block, x, e, context,
  File "C:\Users\45239\Documents\GitHub\stable-diffusion-webui\extensions\sd-webui-modelscope-text2video\scripts\modelscope\t2v_model.py", line 414, in _forward_single
    x = module(x, context)
  File "C:\Users\45239\Documents\GitHub\stable-diffusion-webui\venv\lib\site-packages\torch\nn\modules\module.py", line 1194, in _call_impl
    return forward_call(*input, **kwargs)
  File "C:\Users\45239\Documents\GitHub\stable-diffusion-webui\extensions\sd-webui-modelscope-text2video\scripts\modelscope\t2v_model.py", line 665, in forward
    x = block(x)
  File "C:\Users\45239\Documents\GitHub\stable-diffusion-webui\venv\lib\site-packages\torch\nn\modules\module.py", line 1194, in _call_impl
    return forward_call(*input, **kwargs)
  File "C:\Users\45239\Documents\GitHub\stable-diffusion-webui\extensions\sd-webui-modelscope-text2video\scripts\modelscope\t2v_model.py", line 732, in forward
    x = self.attn1(
  File "C:\Users\45239\Documents\GitHub\stable-diffusion-webui\venv\lib\site-packages\torch\nn\modules\module.py", line 1194, in _call_impl
    return forward_call(*input, **kwargs)
  File "C:\Users\45239\Documents\GitHub\stable-diffusion-webui\extensions\sd-webui-modelscope-text2video\scripts\modelscope\t2v_model.py", line 504, in forward
    sim.masked_fill_(~mask, max_neg_value)
TypeError: bad operand type for unary ~: 'NoneType'
Exception occurred: bad operand type for unary ~: 'NoneType'

### Additional information

_No response_

[kabachuha]

Thanks for the report!

[kabachuha]

While it happened in vanilla mode, I'd recommend moving to Torch2 for better performance

[tin2tin]

Thanks for the quick reply. I'm sorry for my ignorance, but how do I move to Torch2?

[kabachuha]

@tin2tin https://medium.com/@inzaniak/updating-automatic1111-stable-diffusion-webui-to-torch-2-for-amazing-performance-50366dcc9bc1