Open nepoyasnit opened 1 month ago
We don't strict audio sample for the fixed length, so than longer sample than more memory will be allocated for pre-processing stage.
We don't strict audio sample for the fixed length, so than longer sample than more memory will be allocated for pre-processing stage.
Yes, but it seems to me that this memory should be cleared. It is strange that after processing we whatever have a lot of CUDA cache:( Maybe you have any ideas how to clean it? Also, when i use torch.cuda.empty_cache(), the inference time is increasing
Memory allocation is slow operation. Memory layout pre-allocation is an optimization trick to make inference faster. When you call empty_cache() you destroy this optimization.
but it is increasing without any limit, and also it is strange, when we have ~9 Gb CUDA cache with only 3 allocated
in my case, with requests CUDA cache is increasing to 17Gbs, while 3Gbs allocated
I've run your pipeline from repo and saw that CUDA memory grows rapidly from ~3 Gbs to ~12Gbs. If i put small audio, it is also increasing, but not so much.
CUDA memory when i put small (1 second) audio file:![image](https://github.com/user-attachments/assets/c42501f3-2688-4ba0-a315-dd7ec9a0901c)
CUDA memory after putting long audio file (2.5 minutes):![image](https://github.com/user-attachments/assets/166c6228-88d4-40e8-afd8-7da7c82550ef)
Also, I've checked torch.cuda.memory_allocated() and it was constant, but torch.cuda.memory_cached() was increasing. Maybe you can explain me, why CUDA cache is growing?