experiment

[li1u]

I encountered an issue when rendering my own data using mitsuba_neus.py. When I used the result of neus2(not nueus), it reminded me at line 70 of " tmesh. compute_uvalas (tex size)" that tex size is an inappropriate parameter. Perhaps because the model comes with a texture, the original texture needs to be removed. When I use a model without color textures for rendering, the result is only black and white colors. What is the possible reason for this? My camera's internal and external reference settings are incorrect? What if this method requires the input model to remove colors while retaining a structure of the original colors for easier mapping? How should the input model of this method be handled specifically? Below are my rendering models, texture results(predicted_albedo.png), and images. I only added illum2 mao_Kd predicted-albedo.png in the predicted_mesh.mtl file. Is there anything else I missed here? I am very eager to solve this problem now. Looking forward to your reply.

[li1u]

Hello, I'm very sorry to disturb you. I tried another object, but found that its predicted-albedo image is completely black. May I ask how your dataset was specifically deployed?

[benjaminum]

This can be a problem with the camera extrinsics or intrinsics. Did you have a look at the definition of the camera parameter txt files?

[li1u]

Yes, I tested two days ago and confirmed that it should be an issue with the external reference (I processed and compared your data and found that there was not much difference in the camera intrinsics, but the camera extrinsics was very different). Did you handle your camera extrinsics in any way? I extracted it through the colmap visualization window, using PINHOLE as the option, and all other options are default. Then, another question is, when using my own model and your parameters to render the Albedo image, the texture is correct, but the color is gray. I wonder if you have processed the model through Blender? I opened the textured OBJ model directly with Blender, and then removed its color and texture information during export. Do you have any options directly on the visualization interface? Can your colmap_database.py and blender_utils.py files help me solve this problem? Thank you very much.

[li1u]

Hello, the entire process of my camera extrinsics is extracted through colmap -- colmap2nerf is converted to camera extrinsics . There are no other operations besides this, and I failed to run the neus2 method using your parameters. May I ask if your extrinsic parameters are c2w or w2c? Thank you.

[benjaminum]

Extrinsics are w2c. There is no need to convert the colmap camera poses to nerf format

[li1u]

Yes, I made a basic mistake during this process. The external parameters were extracted using tools in nerf, but unfortunately, I forgot to convert them back. Additionally, I failed to convert the c2w format external parameters to w2c. Unfortunately, a new issue arose when I applied this method to human subjects. While it performed well with objects, the results became confusing with humans. Have you had experience conducting experiments on human subjects? What considerations do you believe are crucial when conducting experiments involving the human body? I sincerely apologize for inconveniencing you, and I truly appreciate your response amidst your busy schedule. Thank you.

[benjaminum]

Did the person move while taking the input photos or should all images match the generated mesh?

[li1u]

Yeah, I feel like there might be some issues with the model. I conducted experiments on a human model and the results were good. Do you know how to handle cracks? Thank you very much?

[benjaminum]

We don't have a solution for the visible seams yet. Stronger regularization and more iterations maybe can help to reduce the problem.