search

NicolasHug / Iterable-Datasets-what-could-go-wrong

3 stars 1 forks source link

This is awesome, thank you! #1

Open lparolari opened 6 days ago

lparolari commented 6 days ago

I just wanted to thank you for your work!

Recently, I worked with a dataset that randomly sampled clips from videos. Since each example was selected randomly, I thought it made sense to remove the concept of indexing. For instance, running ds[0] twice would return two different samples. Last month, I decided to switch from a map-style dataset to an iterable one.

But after seeing your notebook, I’m reconsidering—maybe I should go back! 😂

BTW, I saw that you are working on torchcodec which I didn't know it existed, but seems very interesting and quite fit my use case. Do you have some kind of roadmap for that project? Do you plan to release a stable or so version?

Thank you again!

NicolasHug commented 6 days ago

Hi @lparolari , thank you!

Yes, the first "official" release of torchcodec is out now (0.1.0). You can find the docs here: https://pytorch.org/torchcodec/stable/index.html

We still have a lot of features on our backlog but we think it's already usable. We'll publish an announcement on twitter / blog post etc. in the next few days :)