search

Wi-sc / GenMesh

The is the released codes for Single-view 3D Mesh Reconstruction for Seen and Unseen Categories
19 stars 1 forks source link

Preparing Input Data and Handling OBJ Format for GenMesh Training #3

Open MHassan1122 opened 4 months ago

MHassan1122 commented 4 months ago

Hello Dear @Wi-sc Firstly, I would like to express my sincere appreciation for your amazing work on the GenMesh model. It's an impressive piece of work and I'm excited about the potential it has in 3D reconstruction. I am Hassan Master's student at Yuan Ze University Taiwan.

I'm currently looking to prepare input data for the model and had a few queries I was hoping you could help me with:

Data Preparation: Could you provide some guidance on how to properly prepare input data for GenMesh? Specifically, I'm interested in the steps needed to format my dataset correctly to be compatible with the model's expectations.

OBJ Format Compatibility: If anyone has point cloud data in OBJ format or voxalized data. Is it possible to train GenMesh directly using this data? If a conversion is necessary, could you recommend the best practice for this process?

Intrinsic Views Understanding: I'm curious about how GenMesh understands the intrinsic views of objects, particularly for unseen objects during inference or testing, as well as for seen objects during training. How does the model infer this information?

Train for another dataset: Is it possible to train the GenMesh from scratch for other categories?

Thank you in advance for your time and assistance. The answers to these questions will be incredibly helpful for my ongoing project using GenMesh.

Best regards, Hassan

Wi-sc commented 4 months ago

Hi @MHassan1122

Thanks for your interest in our work.

  1. Data Preparation: You need the rendered image and surface points to train the model. For rendering, I used blender here as 3D-R2N2. For points, there are lots of packages such as pytorch3d.

  2. Format: Point cloud and voxel data both work here. But we train the model using Chamfer Distance, so if you want to use voxel, you need to sample surface points from that and save them as point clouds.

  3. Intrinsic parameters: In our work, we assume the intrinsic parameters are the same and the default both for training and testing. But you can set it if you know the actual parameters.

  4. Yes, of course, you can train it from scratch.

Hope you enjoy playing our work. If you have any questions just let me know.

Best, Yang

MHassan1122 commented 4 months ago

Dear @Wi-sc , Thanks for your kind response. Can you please tell me about the data preparation step. How I can (any specific steps to follow to ) prepare the input data if I have obj or .ply formats? sorry for inconvenience.

Wi-sc commented 4 months ago

Of course.

  1. Normalize the object into a unit sphere (radius=0.5) according to vertex coordinates. This step is to make sure the object size is easy to deform from the sphere template.
  2. Sample 10k points from the surface.
  3. Save the sampled point clouds to any format you want such as .npz, .mat, or anything else.