Is there any documentation to create a custom dataset?

[HiroIshida]

lerobot/examples elaborates on how to load and train using the existing dataset on the hugging face repos. Rather I'd like to know how to turn self-collected data into the dataset. So, I'd like to if there is some documentation for that.

[RochMollero]

Same here. I have my data ready but the datasets class seem rather complex to instantiate, so one (or more depending on the number of camera for examples) examples would be very nice.

[TheArtificialOutsider]

same here！I really need an example.

[zwbx]

same question!

[Cadene]

@TheArtificialOutsider @zwbx @RochMollero Got it! We will address this issue very soon, and simplify stuff ;)

Any chance you could provide a very short sample of the datasets in the comment?

In the meantime, a few pointers and ressources:

README:

• https://github.com/huggingface/lerobot?tab=readme-ov-file#the-lerobotdataset-format

See how we use from_preloaded:

• https://github.com/huggingface/lerobot/blob/01f8cede0b5f1c16330205b35f4391939e11cb3e/lerobot/scripts/control_robot.py#L336-L344 (see dataset.stats = stats to use it directly after)
• https://github.com/huggingface/lerobot/blob/main/lerobot/scripts/push_dataset_to_hub.py#L247-L258

See the content of these files to instantiate the hf_dataset, encode the videos, or store frames, etc.

• from hdf5: https://github.com/huggingface/lerobot/blob/main/lerobot/common/datasets/push_dataset_to_hub/aloha_hdf5_format.py#L209-L211
• from zarr: https://github.com/huggingface/lerobot/blob/main/lerobot/common/datasets/push_dataset_to_hub/pusht_zarr_format.py#L257-L259
• from parquet: https://github.com/huggingface/lerobot/blob/main/lerobot/common/datasets/push_dataset_to_hub/dora_parquet_format.py#L213-L215
• from pickle: https://github.com/huggingface/lerobot/blob/main/lerobot/common/datasets/push_dataset_to_hub/xarm_pkl_format.py#L176-L178

[Cadene]

cc @michel-aractingi for visibility ;)

[zwbx]

Hi, thanks for your attention to this matter. I am using RLbench dataset now. I have raw data now, containing image observations, actions. how can I organize them and transfer them to the hf dataset?

Get Outlook for iOShttps://aka.ms/o0ukef

---

From: Remi @.> Sent: Thursday, July 11, 2024 7:08:41 AM To: huggingface/lerobot @.> Cc: Wenbo Zhang @.>; Mention @.> Subject: Re: [huggingface/lerobot] Is there any documentation to create a custom dataset? (Issue #304)

CAUTION: External email. Only click on links or open attachments from trusted senders.

---

cc @michel-aractingihttps://github.com/michel-aractingi for visibility ;)

— Reply to this email directly, view it on GitHubhttps://github.com/huggingface/lerobot/issues/304#issuecomment-2221560936, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AJFDSL3NMJK2265XNQKPZDDZLWSWDAVCNFSM6AAAAABKL436HWVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDEMRRGU3DAOJTGY. You are receiving this because you were mentioned.Message ID: @.***>

[x2ss]

It would be even better if there were tutorials on how to train using custom data from the gym simulation environment.

[Cadene]

Hey there, we are still working on a simplification of the dataset class + upload to hub + tutorial! :) Sorry if it's taking some time!

Unfortunately, the only option now is to get familiar with: https://github.com/huggingface/lerobot/blob/main/lerobot/scripts/push_dataset_to_hub.py

See some example commands in header. You can eventually adapt one of them to your dataset format. If you have issue understanding the code, reach out to us on discord #help channel