Rendering results filled with floating objects

[SalenGit]

First of all, thank you for your work in making it possible to render unbounded scenes quickly. But I found a strange phenomenon in my experiments, when my camera position to be rendered is far from the subject (compared to the training set), the rendered image is filled with floating smoke.

Specifically, the subject of the image appears to have been modeled correctly, as the rendered image has more than sufficient local detail. However, the subject is shrouded by a large amount of floating objects causing the results to look very poor.

My training set images look roughly like this:

And the rendered result looks like this,

I speculate that the current version does not implement distortion loss in mip-nerf360, but I see that there is an experimental stage of distortion loss in your work, can you please tell me how to add it to the training?

These are just some of my guesses for such results, do you have any valuable suggestion for this?

[liruilong940607]

Hi I think in this case you should not use the auto aabb, which is computed from the min max camera locations. You might want to set up a AABB that is suitable for your scene.

[SalenGit]

Thanks to your reply. However, I'm using unbounded mode, their shouldn't have aabb box to limit the reconstruction area.

-------- 原始邮件 -------- 发件人： "Ruilong Li(李瑞龙)" @.> 日期： 2022年12月30日周五 晚上10:05 收件人： KAIR-BAIR/nerfacc @.> 抄送： 田杰 @.>, Author @.> 主 题： Re: [KAIR-BAIR/nerfacc] Rendering results filled with floating objects (Issue #134)

Hi I think in this case you should not use the auto aabb, which is computed from the min max camera locations. You might want to set up a AABB that is suitable for your scene.

― Reply to this email directly, view it on GitHubhttps://github.com/KAIR-BAIR/nerfacc/issues/134#issuecomment-1367936823, or unsubscribehttps://github.com/notifications/unsubscribe-auth/A2D6CYQJZ7TTCTVXVBZMPX3WP3TZFANCNFSM6AAAAAATFW5544. You are receiving this because you authored the thread.Message ID: @.***>

[liruilong940607]

Are you using our example script with auto_aabb enabled?:

https://github.com/KAIR-BAIR/nerfacc/blob/1bc23a180eae1bacfd2ed5452b46f2939f776942/examples/train_ngp_nerf.py#L125-L132

[SalenGit]

Yes, I'm using the same training script with auto aabb enabled.

-------- 原始邮件 -------- 发件人： "Ruilong Li(李瑞龙)" @.> 日期： 2023年1月1日周日 晚上9:59 收件人： KAIR-BAIR/nerfacc @.> 抄送： 田杰 @.>, Author @.> 主 题： Re: [KAIR-BAIR/nerfacc] Rendering results filled with floating objects (Issue #134)

Are you using our example script with auto_aabb enabled?:

https://github.com/KAIR-BAIR/nerfacc/blob/1bc23a180eae1bacfd2ed5452b46f2939f776942/examples/train_ngp_nerf.py#L125-L132

― Reply to this email directly, view it on GitHubhttps://github.com/KAIR-BAIR/nerfacc/issues/134#issuecomment-1368448879, or unsubscribehttps://github.com/notifications/unsubscribe-auth/A2D6CYURNVSDYU5VK7E4YFDWQGETHANCNFSM6AAAAAATFW5544. You are receiving this because you authored the thread.Message ID: @.***>

[liruilong940607]

That is what I was saying, you might want to set up a suitable aabb in this case, instead of using this auto aabb which is just the min max of the cameras.

Note the aabb in the case of unbounded scene, is to set a region that we don't do contraction, and everywhere outside this aabb will be contracted (which will have relatively low quality). You would want to set the aabb as the region of interest that your data is capturing.

[SalenGit]

Thanks for your support, really interesting. I will try it ~

-------- 原始邮件 -------- 发件人： "Ruilong Li(李瑞龙)" @.> 日期： 2023年1月2日周一 上午9:40 收件人： KAIR-BAIR/nerfacc @.> 抄送： 田杰 @.>, Author @.> 主 题： Re: [KAIR-BAIR/nerfacc] Rendering results filled with floating objects (Issue #134)

That is what I was saying, you might want to set up a suitable aabb in this case, instead of using this auto aabb which is just the min max of the cameras.

Note the aabb in the case of unbounded scene, is to set a region that we don't do contraction, and everywhere outside this aabb will be contracted (which will have relatively low quality). You would want to set the aabb as the region of interest that your data is capturing.

― Reply to this email directly, view it on GitHubhttps://github.com/KAIR-BAIR/nerfacc/issues/134#issuecomment-1368600655, or unsubscribehttps://github.com/notifications/unsubscribe-auth/A2D6CYUNWU7VRTXXT47R4BDWQIWZPANCNFSM6AAAAAATFW5544. You are receiving this because you authored the thread.Message ID: @.***>

[ttsesm]

@SalenGit did you manage to obtain any better result by modifying the aabb parameter?