search

facebookresearch / pytorch3d

PyTorch3D is FAIR's library of reusable components for deep learning with 3D data
https://pytorch3d.org/
Other
8.53k stars 1.28k forks source link

optimize textures with TexturesUV and save as obj file #1277

Open Jiakui opened 1 year ago

Jiakui commented 1 year ago

Dear author,

Can I optimize textures with TexturesUV with given image , geometry, and camera pose, then save the result as obj file so that I can view the textured mesh in the MeshLab software?

Is the TexturesUV rendering process fully differentiable ?

Thanks !

bottler commented 1 year ago

Can I optimize textures with TexturesUV with given image , geometry, and camera pose, then save the result as obj file so that I can view the textured mesh in the MeshLab software?

Yes, but unfortunately you have to use the save_obj function (see here). If you use the generic IO method it won't save the texture data. This should be fixed. Thank you for pointing this out.

Is the TexturesUV rendering process fully differentiable ?

I don't know what you mean by "fully", but yes it is differentiable where it makes sense, so the inputs determine how differentiable this is. You need to understand roughly what the model does, e.g. to make occlusion differentiable. (For example, gamma and sigma matter, and soft blending is needed.)

Jiakui commented 1 year ago

Thanks so much for your reply.

I have the mesh and the images , camera pose, in principle I can initialize the texture with xatlas, and then load them into pytorch3d.

For TexturesUV, I need maps, faces_uvs, verts_uvs. As what I understand, I can get faces_uvs, verts_uvs from xatlas, but I need to opitimize maps from pytorch3d. So I need to make maps differentiable .

Is my understanding correct?

bottler commented 1 year ago

I don't know xatlas and I can't help with your modelling question. Rendering is differentiable with respect to maps.