search

dmlc / decord

An efficient video loader for deep learning with smart shuffling that's super easy to digest
Apache License 2.0
1.71k stars 147 forks source link

Bug: AVReader memory leak #279

Open rmira-sony opened 9 months ago

rmira-sony commented 9 months ago

Hi,

Long-time user here, love the package, kudos to the contributors :)

I was trying out AVReader since I work with audiovisual data loaders. However, I've found that it leaks memory, aka using it rather than VideoReader results in climbing system memory usage until the program eventually crashes. To be clear, this does not happen with VideoReader using the same code on the same system.

Here's the wandb plot on system memory usage (pink is AVReader, orange is VideoReader): image

To be more precise, the code for the pink line is:

vr = decord.AVReader(self.files[idx]["video"], sample_rate=self.cfg.sampling_rate, ctx=cpu(0))
wav, video = vr.get_batch(range(len(vr)))
wav = torch.cat(wav, dim=-1)

and the code for the orange line is:

vr = decord.VideoReader(self.files[idx]["video"], ctx=cpu(0))
video = vr.get_batch(range(len(vr)))
wav, old_sr = torchaudio.load(self.files[idx]["video"].replace(".mp4", ".wav"))
wav = torchaudio.functional.resample(wav, old_sr, self.cfg.sampling_rate)

Took me a while to diagnose this so hoping it can help solve this issue. Unfortunately, I'm not really familiar enough with the code to suggest a solution via pull request, so for now I'll stick to the VideoReader. Thanks for reading!

PS: This happens with workers>0 and also with workers=0, so the root is probably not related to the usual multiprocessing conflicts in pytorch dataloaders. PPS: I'm using the latest version of decord, torch and torchaudio. I am using a standard torch dataset class and dataloader with 8 workers.

v-iashin commented 6 days ago

cool, thanks for reporting.

does the same thing happen if you use .VideoReader and .AudioReader separately?