ashawkey / nerf2mesh

[ICCV2023] Delicate Textured Mesh Recovery from NeRF via Adaptive Surface Refinement
https://me.kiui.moe/nerf2mesh/
MIT License
882 stars 88 forks source link

image from render_stage1 has a shrinkage/distortion effect #24

Open tianxiaguixin002 opened 1 year ago

tianxiaguixin002 commented 1 year ago

I test the code on garden data, then compare the validation image with origin image, the contents near image corner has a shrinkage/distortion effect, other data also has this issue. the validation image is generated by differentiable rendering from nvdiffrast, so why these issue happend ? is it a defect of differentiable rendering ? compare image in the following gif. thanks for apply. Peek 2023-04-18 18-59

ashawkey commented 1 year ago

@tianxiaguixin002 Hi, from the error map between GT and Pred renderings, I cannot tell if this is "shrinkage", maybe it's just because the error at edges are higher? ngp_stage1_ep0063_0003_error_22 02

tianxiaguixin002 commented 1 year ago

@ashawkey Yes, the shrinkage phenomenon is higher at edges, it maybe related to renderings error map. In the code, the differentiable rendering image related to mvp matrix, mesh, and color mlp model, so do you know which factor cause these issue ? maybe is the deviation of mvp matrix ?