There is a big difference between rendering and meshing

[zh-en520]

I trained the lego dataset, watched the rendering effect and the mesh effect in the GUI, and found that there is a big difference between the two. Is it because the internal algorithm of "mesh it" is not accurate enough. How can I make "mesh it" achieve the same good effect as rendering.

https://user-images.githubusercontent.com/51226297/186382565-7dc72303-e734-4246-9f73-36af266c85b4.mp4

[PascalHbr]

I am also interested in this!

[saunair]

Could be an artifact of the rasterization for mesh, but here's are my reasons why NeRF rendering is better than Mesh.

1. A mesh won't contain the view-dependent radiance changes. I.e when your camera moves, the color of each point in space is going to remain constant if it is a mesh, unlike a NeRF
2. A mesh has limited resolution (This comes to the rasterization). A NeRF in theory has infinite resolution, as we're applying trilinear interpolation between the features.

[zh-en520]

@saunair Thanks! perhaps you're right. I don't know what you're doing with instant_ngp, I need the mesh generated by NGP to do something interesting in other 3D software. It's a pity that the quality of the mesh I get now is terrible. Do you have any good ways to improve it?

[codgas]

I also have the same problem , using the GUI it looks really good in 3d, but when I use instant-ngp/scripts/run.py to get the mesh it is really bad and it is very sensitive to the aabb_scale , I found the best one is 4 for some reason ? but still it is a very low quality compared to the GUI , any one got an idea ?

[zh-en520]

Possibly, the mesh tool given in instant-ngp is only used for functional verification, and there is still a gap between the algorithms that focus on optimizing meshing. I also tested this project from NVIDIA https://nvlabs.github.io/nvdiffrec. To be honest, it meshes better than NPG, but not by much. Its disadvantage is that it is slow. There are some other mesh tools, such as meshRoom, mvsNet* and so on. instant-NGP is in my opinion the fastest open source tool for training and I have high hopes for this project. I hope the author can optimize the mesh, thank you.

[StefanMenzel]

for reference: I have a Agisoft Metashape license and was running the images through this 'classic' photogrammetry tool at highest settings. The result is not that much better. I believe the small circular structures that are very prominent in this Lego model are simply a rather ungrateful thing to turn into a mesh, as opposed to rectangular structures that just have 8 vertices each. Maybe that helps.

Photo from image set Agisoft Metascan - Dense Cloud Agisoft Metascan - Clay Model Agisoft Metascan - Vertex Colors Agisoft Metascan - Textured Mesh