Is there any way to get the Texture from a PngImageFile object?

[fishbotics]

I'm trying to visualize a mesh that I loaded through Trimesh. It's easy to get the triangular mesh geometry (Meshcat just needs the vertices and faces) but I'm also trying to visualize the textures. For textured meshes, Trimesh uses an object called trimesh.visual.texture.TextureVisuals, which is instantiated with not the png file path but the PIL data as loaded from Pillow.

In Meshcat, the expectation is that I have the png file name, which was lost along the way when loading from Trimesh. Is there any other way to do this conversion?

I know this is kind of an esoteric question and seems sort of silly to be crossing back and forth between mesh libraries, but I find that it's helpful to be able to do mesh manipulation in Trimesh and then visualize complex things in Meshcat.

[fishbotics]

After some more reading, it seems like the easiest thing to do might be to dump the trimesh object into a BytesIO stream and then load it as an ObjFile, but there is no current support for loading textures from obj files (see https://github.com/meshcat-dev/meshcat-python/issues/27)

Another strategy I tried was to take the PIL texture object, dump it into a BytesIO buffer, gran the whole buffer, and then try to instantiate a meshcat.geometry.PngImage object directly, but this also didn't seem to work. Maybe the way I'm converting between objects wrong.

Basically I did the following:

im = mesh.visual.material.image
if im is not None:
    buffer = BytesIO()
    im.save(buffer, format="PNG")
    binary_image_data = buffer.getvalue()
    image = meshcat.geometry.PngImage(binary_image_data)
    material = meshcat.geometry.MeshPhongMaterial(
        map=meshcat.geometry.ImageTexture(image=image)
    )

But this doesn't seem to work with my PNG. In the original OBJ file, I think there are UV coordinates specified to map the texture to the file. Is there something I need to do to make that mapping work?

[kielnino]

Your approach looks right to me. Have you tried to save the png to an file and load it with

material = g.MeshLambertMaterial(map=g.ImageTexture(image=g.PngImage.from_file(path_to_texture_file)))

Another problem may be a missing MTL-File. I often see a combination of a .obj file containing the mesh, a .mtl file containing the mapping, and finally the texture itself.