search

pytorch / vision

Datasets, Transforms and Models specific to Computer Vision
https://pytorch.org/vision
BSD 3-Clause "New" or "Revised" License
16.07k stars 6.94k forks source link

Read videos from memory #6603

Open vedantroy opened 2 years ago

vedantroy commented 2 years ago

🚀 The feature

I have a lot of small videos (3 million video files), in an attempt to reduce disk bottleneck, I store all the videos in LMDB using: https://lmdb.readthedocs.io/en/release/. I would love the ability to read videos from memory.

Motivation, pitch

My training (8xA100 GPUs) process is data starved. I'm pretty sure the issue is disk throughput, hence my usage of LMDB. Allowing torchvision to read videos from memory would help me use LMDB for storing binary files.

Alternatives

I am about to try manually doing this using pyav.

Additional context

No response

cc @pmeier

pmeier commented 2 years ago

Could you clarify what you mean by "read videos from memory"? Do you want to a dataset that allows you to read data from a LMDB?

If yes, you can have a look at the LSUN dataset that stores images inside multiple LMDBs. There is probably room for optimization, but this should get you started.

YosuaMichael commented 2 years ago

cc @jdsgomes

jdsgomes commented 2 years ago

Hi @vedantroy thank you for the feature request. I am actually planning to add this functionality to our VideoReader. I will keep you posted with progress but the ETA is about 2 weeks.