Question for the code of the DPT

[SYSUykLin]

Hello: Great paper. I saw that u utilize the pre-trained Dense Prediction Transformer (DPT) model for zero-shot monocular depth estimation, but I can not find that in the code. Could u specify that? Thx

[zehaozhu]

Hi, the monocular depth of input images is estimated at https://github.com/VITA-Group/FSGS/blob/main/utils/camera_utils.py#L49. We also predict the monocular depth of pseudo views at https://github.com/VITA-Group/FSGS/blob/main/train.py#L124

[SYSUykLin]

Thanks your reply, love u

[SYSUykLin]

Hello, I have a question for your code. depth_loss_pseudo = (1 - pearson_corrcoef(rendered_depth_pseudo, -midas_depth_pseudo)).mean() Why the midas_depth_pseudo is minus? Thx

[zehaozhu]

Hi,

The depth estimated by midas is the inverse depth maps.( https://pytorch.org/hub/intelisl_midas_v2/) So we need a negative function to transform them back. So the logic applies for https://github.com/VITA-Group/FSGS/blob/main/train.py#L109.

Best, Zehao

On Mon, Dec 4, 2023 at 11:22 AM bbab @.***> wrote:

Hello, I have a question for your code. depth_loss_pseudo = (1 - pearson_corrcoef(rendered_depth_pseudo, -midas_depth_pseudo)).mean() Why the midas_depth_pseudo is minus? Thx

— Reply to this email directly, view it on GitHub https://github.com/VITA-Group/FSGS/issues/1#issuecomment-1839120266, or unsubscribe https://github.com/notifications/unsubscribe-auth/APHDBP2A2CFQZ7TH36VAROLYHYBG3AVCNFSM6AAAAABAFLWBB2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQMZZGEZDAMRWGY . You are receiving this because you commented.Message ID: @.***>

[cuttini]

Hi, why in the depth_loss calculation you add +200 on the midas_depth? depth_loss = min( (1 - pearson_corrcoef( - midas_depth, rendered_depth)), (1 - pearson_corrcoef(1 / (midas_depth + 200.), rendered_depth)) ) https://github.com/VITA-Group/FSGS/blob/36ac717ac5997b6a3dc11b47b31d85ba7e80232b/train.py#L107C24-L107C89

Best, Davide

[zehaozhu]

Hi,

We add 200 to make the computation more stable since sometimes midas_depth may be less than zero.

Best, Zehao

On Sun, Dec 31, 2023 at 9:41 AM cuttini @.***> wrote:

Hi, why in the depth_loss calculation you add +200 on the midas_depth? depth_loss = min( (1 - pearson_corrcoef( - midas_depth, rendered_depth)), (1 - pearson_corrcoef(1 / (midas_depth + 200.), rendered_depth)) )

https://github.com/VITA-Group/FSGS/blob/36ac717ac5997b6a3dc11b47b31d85ba7e80232b/train.py#L107C24-L107C89 http://url

Best, Davide

— Reply to this email directly, view it on GitHub https://github.com/VITA-Group/FSGS/issues/1#issuecomment-1872978195, or unsubscribe https://github.com/notifications/unsubscribe-auth/APHDBP2LNV5E2ODVCQVUOILYMGBSPAVCNFSM6AAAAABAFLWBB2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQNZSHE3TQMJZGU . You are receiving this because you commented.Message ID: @.***>