postech-ami / MultiTalk

[INTERSPEECH'24] Official repository for "MultiTalk: Enhancing 3D Talking Head Generation Across Languages with Multilingual Video Dataset"
75 stars 8 forks source link

about CUDA out of memory,when test 3min audio #2

Open zhanghongyong123456 opened 3 months ago

zhanghongyong123456 commented 3 months ago

Test two minutes audio prompt cuda memory is full, I look at the error found is audio feature extraction error, how to achieve the feature extraction of c long video main() File "main/demo_dir.py", line 135, in main test(model, cfg.demo_wav_dir_path, save_folder, condition, subject) File "main/demo_dir.py", line 179, in test prediction = model.predict(audio_feature, template_face, one_hot) File "D:\2023project\project\0win_os\05speech_Pro\MultiTalk\models\stage2.py", line 123, in predict hidden_states = self.audio_encoder(audio, self.dataset).last_hidden_state File "H:\Anaconda3\envs\multitalk\lib\site-packages\torch\nn\modules\module.py", line 1130, in _call_impl return forward_call(*input, kwargs) File "D:\2023project\project\0win_os\05speech_Pro\MultiTalk\models\lib\wav2vec.py", line 147, in forward encoder_outputs = self.encoder( File "H:\Anaconda3\envs\multitalk\lib\site-packages\torch\nn\modules\module.py", line 1130, in _call_impl return forward_call(*input, *kwargs) File "H:\Anaconda3\envs\multitalk\lib\site-packages\transformers\models\wav2vec2\modeling_wav2vec2.py", line 645, in forward layer_outputs = layer( File "H:\Anaconda3\envs\multitalk\lib\site-packages\torch\nn\modules\module.py", line 1130, in _call_impl return forward_call(input, kwargs) File "H:\Anaconda3\envs\multitalk\lib\site-packages\transformers\models\wav2vec2\modeling_wav2vec2.py", line 493, in forward hidden_states, attnweights, = self.attention( File "H:\Anaconda3\envs\multitalk\lib\site-packages\torch\nn\modules\module.py", line 1130, in _call_impl return forward_call(*input, **kwargs) File "H:\Anaconda3\envs\multitalk\lib\site-packages\transformers\models\wav2vec2\modeling_wav2vec2.py", line 372, in forward attn_weights = F.softmax(attn_weights, dim=-1) File "H:\Anaconda3\envs\multitalk\lib\site-packages\torch\nn\functional.py", line 1834, in softmax ret = input.softmax(dim) RuntimeError: CUDA out of memory. Tried to allocate 13.41 GiB (GPU 0; 48.00 GiB total capacity; 96.36 GiB already allocated; 0 bytes free; 97.79 GiB reserved in total by PyTorch) If reserved memory is >> allocated memory try setting max_split_size_mb to avoid fragmentation. See documentation for Memory Management and PYTORCH_CUDA_ALLOC_CONF