Is it possible to add cone to decorative geometry primitives (to visualise friction cones)?

[v-r-a]

Hello,

I'm a research scholar pursuing PhD in humanoid robots. I use MuJoCo (2.3.1) mostly in C/C++.

I have successfully drawn various decorative geometries in the scene, such as lines, arrows, capsules, cylinders etc. They help a lot in trajectory tracking/ control experiments.

I wish there were a built-in elliptical cone primitive to visualize friction cones.

I would appreciate any guidance that would give me a starting point in trying this out myself. My knowledge of C/C++ and GLFW is pretty basic.

I am aware that we have more flexibility in drawing objects in Python.

Thank you

[yuvaltassa]

This is a reasonable request. I'm not sure if this is something we'll want to add to the default contact force visualisation, but adding a cone visual primitive that you could add yourself to the scene will be very easy and the right starting point anyway.

I hope to get around to it in January.

Are you imagining a closed or open cone? (with or without the base?)

What about direction? For world-body collisions the cone should point out of the world, like the current force visualisation, but what about body-body contacts? Are you imagining a double-sided cone? (A bow-tie shape)

[v-r-a]

I prefer an open cone (w/o base). It would not obstruct part geometries or so. Also, since I'm thinking about friction cones, the base does not represent anything.

I had the same dilemma in my mind about whether to request a double cone or not. A double cone would be a more generalised feature. Even for world-body contact, the lower cone would just be left unseen behind the ground. I'm not sure about how front-back is decided in rendering.

[yuvaltassa]

Let's start with a single cone (this is actually the more general option, you can always put two).

I hope to have both closed and open versions so you can choose, but open might be problematic in terms of face culling. We'll see.

I'll ping back here when it works.

[v-r-a]

Hello! I was just wondering if there's been any update on this.

[yuvaltassa]

TBH, it's never the case that the most productive thing to do is this. You're super welcome to add this yourself. If you think you'd be up for a PR I'd be happy to guide you and review it.

[karthyyy]

@v-r-a ,

Whether you found any solution ? I am trying with python.

Regards Karthik

[v-r-a]

Nope, I have no update on it.

I can give it a try/someone with better expertise can give it a try if @yuvaltassa outlines the steps. I believe one can tweak the arrow geometry to get an open cone.

[yuvaltassa]

You want to add an mjGEOM_CONE enum and another case in this switch statement. That's basically it.

After you do that and you get nice cones by manually adding them to the mjvScene (I do this in one of the last cells of the tutorial colab), I can easily hook this up to the friction cone visualization.

[yuvaltassa]

Sorry, reopening.

[v-r-a]

Specs for cone (in the notation of mjv_initGeom): pos: (x,y,z) corresponds to the cone tip size: (a,b,h) elliptical cross-section defined by a and b. Cone height: h mat: Rotation matrix to align to the cone's local frame (positive Z axis: from cone apex to cone base

The existing cone is set to be rendered along with a cylinder for building an arrow. The following seems to be appropriate for rendering a cone alone:

case mjGEOM_CONE:                           // cone
    glScalef(size[0], size[1], size[2]);
    glRotatef(180.0f, 1.0f, 0.0f, 0.0f);
    glTranslatef(0.0, 0.0, -size[2]);
    glCallList(con->baseBuiltin + mjrCONE);
    break;

The output seems to be as expected: The sphere is plotted at pos The central axis is (1,1,1) General elliptical cone

[yuvaltassa]

Thanks @v-r-a. I'm playing with this now and it appears to be wrong... the following works for me though. Could you please confirm/debug?

  case mjGEOM_CONE:                           // cone
      glTranslatef(0.0, 0.0, size[2]);  
      glScalef(size[0], size[1], size[2]);
      glRotatef(180.0f, 1.0f, 0.0f, 0.0f);
      glCallList(con->baseBuiltin + mjrCONE);
      break;

[yuvaltassa]

@v-r-a I'm pretty sure I have this working: cone angle correctly corresponds to friction, including ellipsoidal cone for non-isotropic friction. However, this is only correct for elliptical friction cones. For the default pyramidal cones we need a pyramid :)

Can you try to cook up an mjGEOM_PYRAMID with identical semantics? The unit pyramid we want is strictly circumscribed inside the cone and its corners (clockwise from the top) are at (1, 0, 1), (0, -1, 1), (-1, 0, 1), (0, 1, 1), so in the x-y plane the sides of the pyramid are aligned with x+y=0 and x-y=0 (rather than x and y). Hope this makes sense.

[yuvaltassa]

@v-r-a nevermind, I figured it out, it was amusingly easy. All I needed was to pass nSlice = 4 to the cone() function in render_context.c 🙂 (which I've already modified to support double-sided rendering with an open cone, as discussed above).

I should have the full feature submitted at some point during the week. You guys are lucky I'm bored on a long flight 😝

[v-r-a]

Thanks a lot!

I was wondering if we could see the edges better.

[v-r-a]

Something like this (the sphere is just for debugging)

Tried out something inspired by the mjGEOM_LINEBOX.

case mjGEOM_PYRAMID:                        // pyramid
    glBegin(GL_TRIANGLES);
    glVertex3f(0, 0, 0);
    glVertex3f(size[0], 0, size[2]);
    glVertex3f(0, size[1], size[2]);
    glEnd();
    glBegin(GL_TRIANGLES);
    glVertex3f(0, 0, 0);
    glVertex3f(0, size[1], size[2]);
    glVertex3f(-size[1],0,size[2]);
    glEnd();
    glBegin(GL_TRIANGLES);
    glVertex3f(0, 0, 0);
    glVertex3f(-size[1],0,size[2]);
    glVertex3f(0, -size[1], size[2]);
    glEnd();
    glBegin(GL_TRIANGLES);
    glVertex3f(0, 0, 0);
    glVertex3f(0, -size[1], size[2]);
    glVertex3f(size[0], 0, size[2]);
    glEnd();
    glLineWidth(1.5*con->lineWidth);
    lighting = glIsEnabled(GL_LIGHTING);
    glDisable(GL_LIGHTING);
    glBegin(GL_LINE_LOOP);
    glVertex3f(0, 0, 0);
    glVertex3f(size[0], 0, size[2]);
    glVertex3f(0, size[1], size[2]);
    glEnd();
    glBegin(GL_LINE_LOOP);
    glVertex3f(0, 0, 0);
    glVertex3f(0, size[1], size[2]);
    glVertex3f(-size[1],0,size[2]);
    glEnd();
    glBegin(GL_LINE_LOOP);
    glVertex3f(0, 0, 0);
    glVertex3f(-size[1],0,size[2]);
    glVertex3f(0, -size[1], size[2]);
    glEnd();
    glBegin(GL_LINE_LOOP);
    glVertex3f(0, 0, 0);
    glVertex3f(0, -size[1], size[2]);
    glVertex3f(size[0], 0, size[2]);
    glEnd();
    glLineWidth(con->lineWidth);
    if (lighting) {
      glEnable(GL_LIGHTING);
    }
    break;    

Anyway, thanks a lot!

[yuvaltassa]

Below are images of the test model I'm using. Note the pyramids/cones are open, correctly correspond to <option cone> and non-isotropic friction is correctly visualized (If you didn't know MuJoCo could do this, see issue #67). I think the edges are visible enough? I suspect in your tests your have alpha < 1 which our current renderer is not great at shading...

I played a bit with double-sided cones, it looks pretty awful. My current thinking is that we keep cylinders for non-world contacts and use cones for world contacts, WDYT? Another thing I could do while I'm here (assuming we accept my above proposal to not have cones for non-world) is to use boxes instead of cylinders for non-world contacts, if pyramidal friction cones are used. My problem with this is that boxes are ugly 😝

[v-r-a]

Below are images of the test model I'm using. Note the pyramids/cones are open, correctly correspond to and non-isotropic friction is correctly visualized (If you didn't know MuJoCo could do this, see issue https://github.com/google-deepmind/mujoco/issues/67). I think the edges are visible enough? I suspect in your tests your have alpha < 1 which our current renderer is not great at shading...

Okay, got it. Edges are visible enough. Can we have the central axis of the arrow visible, so that one can visually inspect if the total contact force is within the cone?

I played a bit with double-sided cones, it looks pretty awful. My current thinking is that we keep cylinders for non-world contacts and use cones for world contacts, WDYT? Another thing I could do while I'm here (assuming we accept my above proposal to not have cones for non-world) is to use boxes instead of cylinders for non-world contacts, if pyramidal friction cones are used. My problem with this is that boxes are ugly 😝

I am not sure how someone working in the field of grasp would want to utlise this friction cone visualisation. Maybe maintain the status quo until the requirements are clear? Someone following this discussion may suggest now!

[sachinkundu]

A step backward here but how do you actually get those friction cones? I have a set of contacts and I can get the contact force for that contact using mj_contactForce. However I would like to create the convex hull of the wrench space so would actually like to get the individual friction cone directions assuming pyramidal cones. Do you generate these yourself using the $\mu$ the friction coffecient yourself or is there some place one can read this info from mjData

[yuvaltassa]

@sachinkundu just read the implementation of mj_contactForce...

Also read the implementation of the support functions encodePyramid and decodePyramid