Discussion of the CNN decoder

[tangtaogo]

Hi, Thanks for your great work and the open-source codes! I also tried to use a CNN decoder to decoder rgb from the rendered feature patch and refer to your decoder, however, the resulting RGB image is not smooth and the boundary area between different patches is very obvious. Can you give me some advice, looking forward to your reply!

[mystorm16]

same question

[atonderski]

Hi, could you give some more context? This is not using neurad directly, am i correct? If you generate separate patches and decode them independently you might get these border artefacts. To avoid that we apply the cnn encoder on the entire image during inference (despite training on small patches)

[Crescent-Saturn]

Hi, could you give some more context? This is not using neurad directly, am i correct? If you generate separate patches and decode them independently you might get these border artefacts. To avoid that we apply the cnn encoder on the entire image during inference (despite training on small patches)

As having had the same issue before, I wanted to share that for a better understanding, here is the corresponding source code: https://github.com/georghess/neurad-studio/blob/12745d34426eb732259be35cb7bb4fb04c4df7a1/nerfstudio/models/neurad.py#L624-L669

precisely, L658-L659 shows that the entire feature is fed into CNN decoder.

[tangtaogo]

I got it. Thanks a lot! @Crescent-Saturn @atonderski