search

zenseact / zod

Software Development Kit for the Zenseact Open Dataset (ZOD)
https://zod.zenseact.com
MIT License
92 stars 13 forks source link

Custom Dataset #33

Closed RPalmr closed 6 months ago

RPalmr commented 6 months ago

Is it possible to build a custom size dataset, for example 200GB, potentially using the CLI or another method?

atonderski commented 6 months ago

It is possible, but the functionality is not available out of the box. However, if you go to dropbox you will see that the dataset has been split up into smaller chunks, where the names of each tar indicate which frames/sequences it contains.

If you wanted for example to create a custom dataset of the 10k first frames, you could go to dropbox and download all tars that correspond to these frames. Note that you might need to download some additional frames due to how they were split during packaging. Then you should be able to operate as normal. Just make sure to download the full info files, for the entire dataset (frames or sequences or drives) as those are expected by the devkit. Partial sensor data is not a problem though.

If you end up writing a script for this, feel free to submit a pull request and we might be able to integrate it into the CLI. Otherwise, you can share your custom dataset among coworkers directly, it is not a problem to redistribute the dataset.

Hope this helps!