search

embodied-generalist / embodied-generalist

[ICML 2024] Official code repository for 3D embodied generalist agent LEO
https://embodied-generalist.github.io
MIT License
359 stars 34 forks source link

Thank for your great work, and I have some question. #2

Closed drilistbox closed 7 months ago

drilistbox commented 7 months ago

Thank for your great work, and I have some question：

  1. when will the instruction-tuned LEO weights be released
  2. how to implement various types of simulations (such as Robotic Manipulation, Embodied Navigation ) in the web page ( https://embodied-generalist.github.io/ )?
huangjy-pku commented 7 months ago
  1. Within one week.
  2. The implementation can be found in the repo for our github page.
drilistbox commented 7 months ago

Thanks for your quick reply. But I found the news that "[2024.03] We release the code and data. The embodied AI (EAI) tasks (navigation and manipulation) need further organization and will be released soon.", it seems that navigation and manipulation have not been released.

Besides, I found test_scannet_mask3d.sh in the folder of ./scripts. Directly run the shell with "bash scripts/test_scannet_mask3d", the program finished quickly as following:

bash scripts/test_scannet_mask3d.sh submitted job: 3119752 [2024-03-28 18:35:51,250] - Launched

I apologize for my ignorance, but how can I run the interactive demo or the demo shown on the huggingface(https://huggingface.co/spaces/embodied-generalist/LEO-Demo)?

Thanks a lot!

huangjy-pku commented 7 months ago

Sorry for the misunderstanding. The navigation and manipulation tasks are under re-organization and it takes some time to release.

The shell scripts launch in submitit mode by default. Please check your hardware and choose a proper mode to run.

You can play the interactive demo on the webpage. But if you want to launch your own demo, I am sorry we cannot offer support.

drilistbox commented 7 months ago

Thanks for your reply.

And I have one more question, if I have a robotic arm, how can I control it using code?

huangjy-pku commented 7 months ago

Our work does not involve the control of a robot arm. We use CLIPort for the manipulation task.

drilistbox commented 7 months ago

Thanks a lot!