search

lpiccinelli-eth / UniDepth

Universal Monocular Metric Depth Estimation
Other
648 stars 52 forks source link

Finetuning with Artifacts #67

Open wangqiuwen1006 opened 3 months ago

wangqiuwen1006 commented 3 months ago

During the finetuning process, I noticed that the old version of v2 model might produce some grid-like artifacts, whereas the updated v2 version does not have these artifacts. Could you please explain what specifically might be causing this phenomenon?

lpiccinelli-eth commented 3 months ago

The grid artifacts are due to positional encoding dominating the actual features. Typically it happens in low signal-to-noise regions, like quite texture-less or far away regions. What we changed is how we compute the scale and shift invariant region during training, namely, we compute the statistics only on valid regions (not on the padded regions that appear due to resizing augmentations), this might have pushed the network to ignore the padded regions, which are like the mentioned low SNR regions that usually present those kind of artifacts.