search

open-mmlab / mmdetection

OpenMMLab Detection Toolbox and Benchmark
https://mmdetection.readthedocs.io
Apache License 2.0
29.14k stars 9.39k forks source link

[Feature Request] TorchData example #7546

Open austinmw opened 2 years ago

austinmw commented 2 years ago

Describe the feature

Hi, I know that TorchData is very new, but it would be awesome to see an example of how to create a custom iterable-style datapipe with TorchData for use with this library.

Motivation Iterable-style datasets can be faster and are ideal when all annotations do not fit into memory. TorchData is also going to gain significant traction, so it'd be nice to get ahead of that now.

Related resources https://pytorch.org/data/beta/index.html

Additional context Example:

from torchvision.prototype import datasets
from torchdata.dataloader2 import DataLoader2

train = datasets.load("coco", split="train").shuffle().batch(8).collate()
train_loader = DataLoader2(datapipe=train)
hhaAndroid commented 2 years ago

@austinmw This is great. Can you create a torchdata example PR?

austinmw commented 1 year ago

Hi, it actually seems that the torchvision.prototype is changing too frequently for this to make sense yet. Probably in another ~2 months I can do this. Particularly when they finish revamping the transforms API for object detection.