Implement a widget for 3D triangle mesh rendering

[wkentaro]

I'm trying to render 3D vertices and faces in a widget of glooey.

First of all, to make us on a same page, there is a python library trimesh for loading and visualizing various 3D mesh models. trimesh.viewer.SceneViewer derives pyglet.Window, and what I'm doing now is porting this SceneViewer to a widget of gloooey.

Here is my current implementation of the custom widget after reading the documentation. https://gist.github.com/wkentaro/d8cc30c6634389db02ba53adb7b934b2#file-glooey_3d-py But it won't show any vertex.

Does anyone familiar with extending 2D vertex example in the tutorial to 3D,?

[kalekundert]

I just took a brief look at this. I also couldn't get it to render any vertices, even when I got rid of everything glooey-related. I'll try to look at this in more detail later tonight, but right now my suspicions are either that there's something wrong with lighting, or that the vertices are off-screen somehow.

[kalekundert]

Ok, this should be a working example of a 3D widget:

https://gist.github.com/kalekundert/09966246c0a7a1a4f3f59bfe0f00e6e9

I changed a lot just trying to figure out what was going on, but it turns out that the real problem was that the projection matrix wasn't being set. So instead the default pyglet projection was being used, and it clips any vertex with a z-coordinate not between 0 and 1. You can see how to set the projection (perspective or orthogonal) in the on_resize() function.

In terms of making a custom widget in glooey, there are few things I'd point out:

• Calling _regroup() in the constructor like that is fragile, MyGroup will get overridden if the widget get regrouped later for any reason (e.g. if it's added to a scroll box). It's better to just create the group you need for the vertex lists on the fly in do_regroup() and do_draw(). Maybe it helps to think of self.group as being the widget's group, which is distinct from the vertex list's group.

• There's no need to call self.vertex_list.draw() in do_draw(). The vertex list has been added to the GUI's batch, and the whole batch will be drawn automatically.

• I added a bunch of code (SpinGroup, do_attach(), do_detach(), on_update()) to make the mesh spin. These were helpful to me for debugging, but shouldn't really be necessary for you. Hopefully they're not confusing.

Let me know if that helps, or if you have any other questions!

[wkentaro]

You implement very quick! Thanks. I will work around with your code.

[wkentaro]

Have you tried to set the projection in SceneGroup without overwriting on_resize? https://gist.github.com/wkentaro/ff4b804df77d8292fdb7043c3e2489f9

class SceneGroup(pyglet.graphics.Group):

    def set_state(self):
        self._enable_depth()
        self._enable_color_material()
        self._enable_blending()
        self._enable_smooth_lines()
        self._enable_lighting()
        self._clear_buffers()

        width, height = 640, 480
        glViewport(0, 0, width, height)
        glMatrixMode(GL_PROJECTION)
        glLoadIdentity()
        gluPerspective(60, width / height, 0.01, 1000.0)
        #glOrtho( 0, width, 0, height, -1000.0, 1000.0)
        glMatrixMode(GL_MODELVIEW)

It gives below output:

And also, I've found that the SceneWidget cannot be used with HBox in some reason: https://gist.github.com/wkentaro/3cda5faa6687272f942ca9dfb24bd0f6

    hbox = glooey.HBox()

    mesh = trimesh.creation.annulus(0.5, 1, 0.2)
    #mesh = trimesh.creation.icosahedron()
    #mesh = trimesh.creation.axis(0.1)
    scene = SceneWidget(mesh)
    hbox.add(scene)

    placeholder = glooey.Placeholder(min_width=640, min_height=480)
    hbox.add(placeholder)

    gui.add(hbox)

I will try to fix these problems.

[kalekundert]

Yeah, you can probably setup the projection in the scene group. I didn't think about that before, but I think it'd be a good way to encapsulate all the OpenGL logic in one place. You could also avoid having to hard-code the window size by passing the scene widget to the SceneGroup constructor. In set_state(), you could access the widget's window attribute to get the current size of the window.

There are a few things worth mentioning about the HBox. I assume the error you're getting with the HBox is:

RuntimeError: Gui(id=ec18) is only 640x480, but its children are 1280x480.

What this means is that the HBox wants to be 1280 pixels wide, but the window is only 640 px wide, so the hbox won't fit. The reason the hbox wants to be 1280 px wide is that the placeholder is 640 px wide, and the hbox wants all of its columns to be the same width, so it allocates 640 px for the scene widget as well. If you use hbox.pack(scene) or hbox.add(scene, size=0) to add the scene widget, then the hbox columns won't have to be the same size and the code will run (although the scene will be allocated its minimum width: 0 px). Alternatively, you could make the placeholder smaller.

But the second problem is that the scene widget doesn't actually take its rectangle into account when it's drawing itself. So no matter where the scene widget is positioned by the hbox, it'll draw its 3D vertex lists in the same place. I think the best way to deal with this would be to give SceneGroup a reference to the widget (as mentioned above) and then to have the group translate everything based on widget.rect.bottom_left (or something like that). You might also want to have the SceneGroup clip the scene to the boundaries of the widget using something like this:

    # From `glooey.drawing.stencil.ScissorGroup`
    def set_state(self):
        glPushAttrib(GL_ENABLE_BIT)
        glEnable(GL_SCISSOR_TEST)
        glScissor(
                int(self.rect.left),
                int(self.rect.bottom),
                int(self.rect.width),
                int(self.rect.height),
        )

    def unset_state(self):
        glPopAttrib()

Let me know if that helps!

[wkentaro]

Thanks! All of the information is very helpful.

Now the code is here: https://gist.github.com/wkentaro/ff4b804df77d8292fdb7043c3e2489f9 The main part I changed is

        glPushAttrib(GL_ENABLE_BIT)
        glEnable(GL_SCISSOR_TEST)
        glScissor(
            int(self.rect.left),
            int(self.rect.bottom),
            int(self.rect.width),
            int(self.rect.height),
        )

        left, bottom = int(self.rect.left), int(self.rect.bottom)
        width, height = int(self.rect.width), int(self.rect.height)
        glViewport(left, bottom, width, height)
        glMatrixMode(GL_PROJECTION)
        glLoadIdentity()
        gluPerspective(60, width / height, 0.01, 1000.0)
        glMatrixMode(GL_MODELVIEW)

Now I get below, which looks almost close.

The last problem I am looking at is the completely black region of the left side, where a placeholder should appear. With below change, it looks

diff --git a/glooey_issue_24.py b/glooey_issue_24.py
index ace8067..0fe88ed 100644
--- a/glooey_issue_24.py
+++ b/glooey_issue_24.py
@@ -174,8 +174,8 @@ if __name__ == '__main__':

     mesh = trimesh.creation.annulus(0.5, 1, 0.2)
     scene = SceneWidget(mesh)
-    hbox.add(scene)
-    # hbox.add(glooey.Placeholder(min_width=640, min_height=480))
+    # hbox.add(scene)
+    hbox.add(glooey.Placeholder(min_width=640, min_height=480))

     gui.add(hbox)

If I comment out self._clear_buffers(), the placeholder appears in a moment, but it disappears immediately.

If I comment out like below:

diff --git a/glooey_issue_24.py b/glooey_issue_24.py
index ace8067..f1eede2 100644
--- a/glooey_issue_24.py
+++ b/glooey_issue_24.py
@@ -19,7 +19,7 @@ class SceneGroup(pyglet.graphics.Group):
         self._enable_blending()
         self._enable_smooth_lines()
         self._enable_lighting()
-        self._clear_buffers()
+        # self._clear_buffers()

         glPushAttrib(GL_ENABLE_BIT)
         glEnable(GL_SCISSOR_TEST)
@@ -30,13 +30,13 @@ class SceneGroup(pyglet.graphics.Group):
             int(self.rect.height),
         )

-        left, bottom = int(self.rect.left), int(self.rect.bottom)
-        width, height = int(self.rect.width), int(self.rect.height)
-        glViewport(left, bottom, width, height)
-        glMatrixMode(GL_PROJECTION)
-        glLoadIdentity()
-        gluPerspective(60, width / height, 0.01, 1000.0)
-        glMatrixMode(GL_MODELVIEW)
+        # left, bottom = int(self.rect.left), int(self.rect.bottom)
+        # width, height = int(self.rect.width), int(self.rect.height)
+        # glViewport(left, bottom, width, height)
+        # glMatrixMode(GL_PROJECTION)
+        # glLoadIdentity()
+        # gluPerspective(60, width / height, 0.01, 1000.0)
+        # glMatrixMode(GL_MODELVIEW)

     def unset_state(self):
         glPopAttrib()

It looks like below:

so it seems there are some issues around glClear, gluPerspective, and the OpenGL context in SceneGroup is affecting other widgets as well.

Do you know or have any idea how to encapsulate these OpenGL contexts?

[kalekundert]

I haven't tested anything with code, but my guess is that you have to reset the projection matrix in unset_state(). My instinct would be to use glPushMatrix() and glPopMatrix(). Here are some links talking about the pros and cons of that approach:

https://community.khronos.org/t/glpushmatrix-and-gl-projection/56758/3 https://gamedev.stackexchange.com/questions/93237/glpush-popmatrix-on-projection-matrix https://stackoverflow.com/questions/4269079/mixing-2d-and-3d-in-opengl-using-pyglet

You could also just set a hard-coded orthgraphic projection in unset_state(), but that might make it harder for other people to use your code.

[wkentaro]

Thanks! Now HBox perfectly works (I needed to restore mode, viewport and projection matrix).

https://gist.github.com/wkentaro/ff4b804df77d8292fdb7043c3e2489f9

[kalekundert]

Nice! Let me know if you run into any more issues. I think it might also be worth trying to get something like this merged into the trismesh project (i.e. if not the widget itself, at least something that does all the hard work so the widget can be much simpler), so let me know if that's something you'd be interested in coordinating on.

[wkentaro]

Yeah. I already told the author of trimesh about glooey, and I think he’s interested in this project. So I expect I can make the widget to trimesh.

https://github.com/mikedh/trimesh/issues/368