Open JosselinSomervilleRoberts opened 5 days ago
Hi @JosselinSomervilleRoberts,
Thank you for reaching out. I'm afraid DALI doesn't support the sampling patterns you ask for. What it can do is sample video with constant steps and stride, while in your case, you look for the equal distribution of a fixed number of samples.
Ok thank you for letting me know! Please if this update is made in the future, I would love to hear about it!
Describe the question.
Hi, I just learned about DALI and wanted to ask if it was the correct tool for my use case. I have a dataset of videos and I want to load them in a Dataloader in PyTorch. I work on multiple GPUs.
My pipeline goes like this:
file_name
by accessingfnames[index]
(fnames: List[str]
)file_name
. (the number of frames might be different for each video)T: int
frames I want to extract. This valueT
is a constant and so will be the same for each video but the indexing might differ. (If I wantT=3
frames uniformly sampled in a video of 101 frames it would be[0,50,100]
while it would be[0,100,200]
in a video of 201 frames)T
frames in the video (hopefully without having to decode the entire video)T,C,H,W
Now what I want is the batch version of this in a distributed manner. So a pipeline that gives me some frames of shape
B,T,C,H,W
.I am currently using a custom
DataLoader
currently and in__get_item__(index: int) -> torch.Tensor
I call aload_video(fname: str, rel_indices: np.ndarray) -> torch.Tensor
that can be implemented with different engines (Decord, torchvision.io, ...) which are all too slow. If I understand correctly, the setup of DALI is different as it directly processes batches?Do you think DALI could be useful in my use case and if so how could I implement this? Keep in mind that I am working in a distributed setup with multiple GPUs (and potentially multiple nodes later on) and that the number of frames extracted
T
is significantly smaller that the number of frames available.Thanks!
Check for duplicates