when will other models be added ?

huggingface / lerobot

🤗 LeRobot: Making AI for Robotics more accessible with end-to-end learning

Apache License 2.0

6.54k stars 582 forks source link

when will other models be added ? #259

Closed ssvision closed 2 months ago

ssvision commented 3 months ago

how do i train and visualise my policy in the environment for other aloha and diffusion tasks? the hugging face repo has only 2 models namely diffusion_pusht & act_aloha_sim_transfer_cube_human?

how do i do the same for other tasks such as aloha insertion task ? and other range of tasks

aliberts commented 3 months ago

Hi, we are working on adding several models to the library, namely:

Octo (#209)
VQ-BeT (#166)
TD-MPC is already in our codebase and we did reproduce their results initially but since then we've migrated the xarm env and we've been having compatibility issues because of it (the dataset doesn't exactly produces the same outcomes in the new version of the env), which is why we haven't uploaded a pretrained model for it yet.

We'll also add more documentation & tutorials on how to switch tasks when training a given policy ;)

alexander-soare commented 3 months ago

@ssvision if you train a model for a policy/dataset/env combination we haven't already, and it matches or exceeds a SOTA benchmark, we can add it on the hub! What are you mos interested in? I heard aloha insertion. The reason we haven't added that is that (a) the results are pretty poor on the ACT paper with only 20% success rate, (b) we actually haven't been able to reproduce, falling about 5% short either because of not trying enough seeds or maybe some of these tweaks that still need to go into ACT: https://github.com/huggingface/lerobot/pull/265

alexander-soare commented 2 months ago

Closing due to inactivity. @ssvision if you have further questions, please don't hesitate to re-open.