haosulab / ManiSkill

SAPIEN Manipulation Skill Framework, a GPU parallelized robotics simulator and benchmark
https://maniskill.readthedocs.io/en/latest/
Apache License 2.0
555 stars 97 forks source link

Question: Can I swap simulation robot by a real robot? #343

Open rafaelBauer opened 1 month ago

rafaelBauer commented 1 month ago

Hello,

Would it be possible to have a setup where I could swap a simulated robot by a real robot, or this is not really on the scope here? I didn't dig deep into ManiSkill yet, but since it supports multiple robots I imagine that there is a robot interface in which specific robots specialise, so one could then have an implementation which would convert the software interface into ROS and therefore control a real robot...

For now what I have done is to create an "BaseEnvironment" and an specialisation that wraps a ManiSkill environment so later I can simply create another specialisation with the real robot environment.

Again this is just more an "exploration question", since I first want to have a simulation to verify my algorithm, and later I will test it in the real world. Maybe I would be able to reuse some code and so on.

Kind regards, Rafael

StoneT2000 commented 1 month ago

Hi, what do you mean by swapping a simulated robot by a real robot? Do you mean to create some gym environment here and when you take an action here, it takes the corresponding action on the real robot via ROS?

If that is what you refer to we currently don't have a ROS integration in this library although it certainly would be very welcome (it could be very helpful for system ID to check simulation controller matches behavior of the real world one). We only have some messy internal code we used for our own projects that is not ready for release.

What you suggest by creating a wrapper makes sense. I would also do the same where i create a wrapper class that overrides the step and reset functions to call their respective super functions and also apply thta action via ROS to the real robot and/or wait for the real robot to finish.

StoneT2000 commented 3 days ago

Hi @rafaelBauer do you still have any questions about this?