google-deepmind / mujoco_menagerie

A collection of high-quality models for the MuJoCo physics engine, curated by Google DeepMind.
Other
1.45k stars 202 forks source link

Add LEAP hand. #91

Closed kevinzakka closed 2 months ago

kevinzakka commented 2 months ago

Fixes #84.

leap_hand

kevinzakka commented 2 months ago

cc @Lucky7Chess

Lucky7Chess commented 2 months ago

Yes, the right hand looks good to me @kevinzakka I just loaded it. Maybe we should also add the left hand also from https://leaphand.com/static/cad/urdf/leap_left.zip ?

I'll add a link to your file on menagerie to the LEAP Hand website.

kevinzakka commented 2 months ago

Awesome, thanks so much for fast review @Lucky7Chess! Agree having the left hand would be good. Is it as simple as reflecting the meshes about the z axis? I'm trying to see if we can write a script to generate the left from right to minimize bugs.

kevinzakka commented 2 months ago

I noticed the following differences between the left and right models:

Just want to make sure that these are intentional and wanted to walk through them with you @Lucky7Chess.

left right
Lucky7Chess commented 2 months ago

Sorry for some reason Github didn't notify me of your message.

Yes I wish there was an easy way to mirror but there is not it seems. But I think the model isn't an exact mirror in this case.

None of these three things you mention are intentional actually. The first two is just some CAD software naming I didn't change. And the last is just how my CAD software exports and I never fixed it to a set position. Its just a byproduct of the software I am using to get the URDF. Its a messy mix of Inventor/Onshape.

I guess I don't use the left hand much so I haven't noticed the differences. Its probably better to fix these bugs in the left on Menagerie since that'll be what people use on MuJoCo.

kevinzakka commented 2 months ago

~@Lucky7Chess What does thumb_left_temp_base correspond to? I tried replacing it with pip and adjusting the transform but struggling to match the right hand.~

Was able to figure this out!

kevinzakka commented 2 months ago

@Lucky7Chess One last thing: I want to add an attachment site to each hand so that users can use it to attach the hand to an arm. Do you have a pos/quat that I can use for this for each hand, potentially from CAD? I couldn't find anything from parsing the original URDF.

Lucky7Chess commented 2 months ago

Hmm I actually never thought about adding that info to the URDF/MJCF but that's a really good idea. I can't figure out how to get the coordinates from the URDF in the format you may want but I can get it in my CAD. The center of the mounting holes is where the center of the arm end effector mounting point would contact the hand. As in the photo below that is 37.5mm from the top of the top plate and 59mm from the flat side of the top plate.

image

The top plate is the same for the right hand (the plate you are looking at on the image attached) and the left hand except it is mirrored. So for the left hand it is 37.5 mm from the top and 59mm from the right side which is still the flat side of the top palm.

If this doesn't make sense I can also try to figure this out in URDF somehow? I am unfamiliar with the easiest way to do that. Thank you for your help, you are an expert in this! :)

Lucky7Chess commented 2 months ago

It also looks like I didn't put the mounting holes into the URDF, not sure why. The hand typically mounts to UR5/Xarm/Franka that's why its mounted to the top of the hand to avoid the arm colliding with the table due to the kinematic structure of these arms being designed for grippers that are also top down.

kevinzakka commented 2 months ago

@Lucky7Chess Maybe we can have the mounting plate as part of the URDF? If it's arm specific then maybe not. But I need the coordinates of the center in the palm frame if possible otherwise I can visually guesstimate the location!

Lucky7Chess commented 2 months ago

The plate is arm specific depending on the hole pattern of the arm...so the plate you have on the urdf is just a generic one without any holes. I guess now that I think of it the mounting position could also change slightly depending on the arm you have.

You could leave the mounting point off I guess or just pick a center location on the top of the palm, either way is fine. The point that I sent you is roughly aligned with the middle finger and slightly less than halfway down from the start of the middle finger/top from that image viewpoint.

kevinzakka commented 2 months ago

I think I’ll go with the latter option!

The only thing is that the estimates for left and right are different because the palm meshes seem to have a different coordinate frame. I want to figure out how to make them symmetric 🥲

Lucky7Chess commented 2 months ago

At one point I was trying to make them symmetric also but then I gave up I remember. The CAD -> URDF exporters I have all suck to be honest and there was no easy way for me to do this.

Lucky7Chess commented 2 months ago

I think its okay because as the end user its not really noticable how the meshes are setup.

kevinzakka commented 2 months ago

Thanks @Lucky7Chess, decided not to add an attachment site for now. We can add it in a future PR once we also validate with actual hardware!

kevinzakka commented 2 months ago

@yuvaltassa Ready for import now!

Lucky7Chess commented 2 months ago

Thank you so much @kevinzakka !!!

kevinzakka commented 2 months ago

I didn't do much, thank you for your awesome open-source contributions :)