demo out of memory on vocabulary 'lvis-paco'

[irimsky]

I ran the demo on vocabulary 'lvis-paco' with the command:

python demo/demo.py --config-file configs/joint_in/swinbase_cascade_lvis_paco_pascalpart_partimagenet_inparsed.yaml \ --input input1.jpg input2.jpg input3.jpg \ --output output_image \ --vocabulary lvis_paco \ --confidence-threshold 0.7 \ --opts MODEL.WEIGHTS models/swinbase_cascade_lvis_paco_pascalpart_partimagenet_inparsed.pth VIS.BOX False

then it printed 'Killed' and exited. I switched it to gpu, then it came out with the error:

Traceback (most recent call last): File "/workspace/VLPart/demo/demo.py", line 116, in demo = VisualizationDemo(cfg, args).to("cuda:0") File "/workspace/VLPart/demo/predictor.py", line 124, in init classifier = get_clip_embeddings(self.metadata.thing_classes) File "/workspace/VLPart/demo/predictor.py", line 30, in get_clip_embeddings emb = text_encoder(texts).detach().permute(1, 0).contiguous() File "/root/miniconda3/envs/3s/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1501, in _call_impl return forward_call(*args, kwargs) File "/workspace/VLPart/./vlpart/modeling/text_encoder/text_encoder.py", line 166, in forward features = self.encode_text(text) # B x D File "/workspace/VLPart/./vlpart/modeling/text_encoder/text_encoder.py", line 154, in encode_text x = self.transformer(x) File "/root/miniconda3/envs/3s/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1501, in _call_impl return forward_call(*args, *kwargs) File "/workspace/VLPart/./vlpart/modeling/text_encoder/text_encoder.py", line 61, in forward return self.resblocks(x) File "/root/miniconda3/envs/3s/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1501, in _call_impl return forward_call(args, kwargs) File "/root/miniconda3/envs/3s/lib/python3.10/site-packages/torch/nn/modules/container.py", line 217, in forward input = module(input) File "/root/miniconda3/envs/3s/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1501, in _call_impl return forward_call(*args, *kwargs) File "/workspace/VLPart/./vlpart/modeling/text_encoder/text_encoder.py", line 46, in forward x = x + self.attention(self.ln_1(x)) File "/workspace/VLPart/./vlpart/modeling/text_encoder/text_encoder.py", line 43, in attention return self.attn(x, x, x, need_weights=False, attn_mask=self.attn_mask)[0] File "/root/miniconda3/envs/3s/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1501, in _call_impl return forward_call(args, **kwargs) File "/root/miniconda3/envs/3s/lib/python3.10/site-packages/torch/nn/modules/activation.py", line 1205, in forward attn_output, attn_output_weights = F.multi_head_attention_forward( File "/root/miniconda3/envs/3s/lib/python3.10/site-packages/torch/nn/functional.py", line 5224, in multi_head_attention_forward q, k, v = _in_projection_packed(query, key, value, in_proj_weight, in_proj_bias) File "/root/miniconda3/envs/3s/lib/python3.10/site-packages/torch/nn/functional.py", line 4767, in _in_projection_packed proj = proj.unflatten(-1, (3, E)).unsqueeze(0).transpose(0, -2).squeeze(-2).contiguous() torch.cuda.OutOfMemoryError: CUDA out of memory. Tried to allocate 750.00 MiB (GPU 0; 31.74 GiB total capacity; 29.76 GiB already allocated; 724.12 MiB free; 30.04 GiB reserved in total by PyTorch) If reserved memory is >> allocated memory try setting max_split_size_mb to avoid fragmentation. See documentation for Memory Management and PYTORCH_CUDA_ALLOC_CONF

My GPU has 40GB memory. I want to know on what condition I can run the demo on vocabulary 'lvis-paco'