search

ForMyCat / SparseGS

Codebase for SparseGS paper
Other
103 stars 11 forks source link

Depth loss #7

Closed manurare closed 4 months ago

manurare commented 4 months ago

Hi,

Thanks for sharing this amazing codebase!

I was wondering. Have you tried as depth loss a normal L1 loss. Do you have any insights of why it might or might not work compared with the pearson loss?

ForMyCat commented 4 months ago

The problem is that monocular estimation models only give you relative depth maps, meaning the depth ranges from 0-1. A depth value of 1 from two images may represent different metric depth values. If we simply normalize all the rendered depth maps then do a L1 loss, the difference in variance between views will cause ambiguity. We have not tested this yet, but a possible approach is to use a MLP to find correct and consistent scalings to scale all the relative depth maps to metric values, then fit a pixel-wise loss such as L1/L2. I am curious about the result as well. Feel free to follow up with me if you have some findings.

On Thu, May 23, 2024 at 3:25 AM manurare @.***> wrote:

Hi,

Thanks for sharing this amazing codebase!

I was wondering. Have you tried as depth loss a normal L1 loss. Do you have any insights of why it might or might not work compared with the pearson loss?

— Reply to this email directly, view it on GitHub https://github.com/ForMyCat/SparseGS/issues/7, or unsubscribe https://github.com/notifications/unsubscribe-auth/ALV4NNAYLYZOQUW3BHJRUBTZDW7Y7AVCNFSM6AAAAABIFLEECOVHI2DSMVQWIX3LMV43ASLTON2WKOZSGMYTENJTGQZTKNI . You are receiving this because you are subscribed to this thread.Message ID: @.***>