Questions about the loss computation

[w2kun]

Hi, thanks for releasing the code!

I have several questions after reading the execute_func.py:

• At L78, the DPT is initialized with invert=True, which means the output is already depth. While at L253, a division is performed again to convert the output of DPT to depth. Is there a comment error here, since from what I understand, the code at L254 seems to be correct.
• At L378, the pseudo_depth error is computed between outputs["fromMono_dep"] and outputs[('depth', 0, 0)], which have different scales. Should outputs["fromMono_dep"] be replaced by target_depth at L353, since its scale has been aligned.

Looking forward to your reply.

Best regards!

[choyingw]

That's an error when we switch to NYU's pretrained model. Please uncomment L64-71 and comment L74-81, and the rest is intact. I'll update it

[choyingw]

Just updated. If you use NYU's pretrained model, there is no need to do the inversion since the model's output is directly in meters; otherwise when a general purpose DPT is used (i.e. predicting in disparity space), you need to do inversion

[w2kun]

Thanks for your reply! I will try the updated version. Also, I'm wondering if it's correct to modify the code like the second item says?

[baldgirllulu]

In subsection 3.2 of the paper it is written that“We first estimate rough alignment factors of scale as and shift at from DPT’s output D∗ t to predicted depth Dt by minimizing differences between D¯ ∗ t = asD∗ t + at and Dt with closed-form expressions from the least-square optimization (see the supplementary” My understanding is that the output of DPT needs to be varied linearly and then with the predicted depth Dt to calculate Statistical loss. but in the code directly using the output of DPT and the predicted depth to calculate the ssim value, and I didn't find any program related to the linear variation, is it my misunderstanding? Looking forward to your reply.

Best regards!

[baldgirllulu]

About the second item：I also think scale alignment is needed before computing the loss