Softras to Pytorch3d comparison rendering flow

[BastiaanGroot]

Hi all!

I am currently in the process of converting a current Softras implementation to Pytorch3d. I have a set of 3d deformable meshes (frames) of which i would like to calculate the flow from (i.e. flow = diff(mesh_1 - mesh_1 +1)) The Softras implementation does this quite clever by using the verts of the following frames:

mesh_flow = sr.Mesh(verts[:-1], faces.repeat(int(self.config.data.batch_size) - 1, 1, 1), textures=verts[1:],
                  texture_type="vertex")

rendering_uv = self.renderer_softtex.render_mesh(mesh_flow)

I tried to replicate this using pytorch3d without success:

verts_rgb = verts[1:]
mesh_flow = Meshes(verts=verts[:-1], faces=faces.repeat(self.config.data.batch_size - 1, 1, 1),
                 textures=TexturesVertex(verts_features=verts_rgb))
rendering_uv = self.phong_renderer(mesh_flow, R=R, T=T)

The verts argument is simply a batch of vertices + rgb values without actual uv values torch.Size([2, 2505, 3])

[bottler]

I think you need to work this out step by step yourself based on the data shapes. We can't do this for you.