Odd design in the decoder network

suhangpro commented 5 months ago

Hello! I am trying to understand the decoder (DepthDecoderPP) architecture and see this line of code that seems to make redundant assignments (and therefore overwriting previous values). This line is called multiple times with the same i. Is this by design?

https://github.com/nianticlabs/simplerecon/blob/477aa5b32aa1b93f53abc72828f86023b6e46ce7/modules/networks.py#L92

Also, I find it challenging to get a clear picture of DepthDecoderPP's architecture. Can you help offer some clarifications?

Thanks in advance!

mdfirman commented 5 months ago

Hello! Sorry for the delay, and thanks for your interest.

Yes – you are right that there are some redundent assignments. I believe a small speedup could be made by replacing that line with:

                if i + j == 4:
                    depth_outputs[f"log_depth_pred_s{i}_b1hw"] = self.convs[f"output_{i}"](output)

The depth decoder is based on Unet++ (https://arxiv.org/pdf/1807.10165.pdf). Figure 1 from that paper might be informative.

To help further with understanding the architecture, I have made the following diagram. I hope this helps!

DepthDecoderPP

suhangpro commented 5 months ago

Fantastic! Really appreciate the effort👍

nianticlabs / simplerecon

Odd design in the decoder network #44