facebookresearch / pytorch3d

PyTorch3D is FAIR's library of reusable components for deep learning with 3D data
https://pytorch3d.org/
Other
8.81k stars 1.32k forks source link

How to create a uniform TexturesUV by giving verts and faces? #1773

Closed KzZheng closed 6 months ago

KzZheng commented 7 months ago

Given verts and faces, I want to create a simple mesh with a uniform texture. Here is a simple code:

import torch
from pytorch3d.renderer import TexturesUV
from pytorch3d.structures import Meshes
from pytorch3d.vis.plotly_vis import AxisArgs, plot_batch_individually, plot_scene

device = torch.device("cuda:0")
xyz = [
        -2.1253,
        0,
        1.3118,
        -4.1093,
        0,
        3.1478,
        -4.1093,
        0,
        1.3118,
        -4.1093,
        0,
        3.1478,
        -2.1253,
        0,
        1.3118,
        -2.1253,
        0,
        3.1478
    ]
face_list = [
                0,
                2,
                1,
                3,
                5,
                4
            ]
verts_uvs = torch.tensor([
    [0.5, 0.5],
    [0.5, 0.5],
    [0.5, 0.5],
    [0.5, 0.5],
    [0.5, 0.5],
    [0.5, 0.5],
], device=device)
maps = torch.ones(1, 128, 128, 3, dtype=torch.float32, device=device)
verts = torch.tensor(xyz, device=device).view(-1, 3)
faces = torch.tensor(face_list, device=device).view(-1, 3)
textures = TexturesUV(maps=maps, faces_uvs=faces[None, ...], verts_uvs=verts_uvs[None, ...])
mesh = Meshes(
    verts=[verts.to(device)],   
    faces=[faces.to(device)],
    textures=textures
)
fig = plot_scene({
    "subplot1": {
        "cow_mesh": mesh
    }
})
fig.show()

However, I got the mesh with the default texture (same as set textures=None), shown below. I wonder how to create this TexturesUV correctly. Thanks!

Screenshot from 2024-04-03 18-29-37

PS, I also tried to use TexturesVertex, which can get correct mesh textures, but it seems impossible to render with other meshes (which have TexturesUV) together. It would be very much appreciated if there was a solution to this!

bottler commented 7 months ago

Your code is correct. The PyTorch3D renderer and other parts of PyTorch3D understand your mesh as being white. The plotly_vis functionality does not understand TexturesUV and just ignores it, because plotly itself, as far as I know, does not support a uv map on a mesh plot.

A workaround could be added to plotly_vis to make an approximation (e.g. give a constant color to each face based on the average color in the texture over that face, or even somehow to use a surface plot instead of a mesh plot) but this hasn't been implemented.

The only textures which plotly_vis currently understands are TexturesVertex (which corresponds to vertex colors in plotly) and TexturesAtlas with K equal to 1, i.e. a constant color per face (which corresponds to face colors).