search

shubham-goel / ds

Code for Differentiable Stereopsis. Goel, Gkioxari, Malik 2021
71 stars 7 forks source link

DS does not work well when object is small in image #3

Open xiexh20 opened 2 years ago

xiexh20 commented 2 years ago

Hello,

Thanks for the great work.

I have some data where the object only occupies a relatively small area in the whole image. When I run your method the results are not that good. I suspect it is because there are not many gradients propagate back to deform the mesh. Do you have any suggestions/insights about this? Thank you very much!

Below are the input views and 360 views of reconstructed mesh. kroi-hres-monitor-s1 0-v4_gt_rend kroi-hres-monitor-s1 0-v4_texDepthNormal

shubham-goel commented 2 years ago

Hi, apologies for the late reply! I find this failure a little surprising. It seems like a mesh scaling issue, which is sensitive to hyperparameters but I have some code to rescale input here which should work automatically during data-loading.

Can you please share your data with me? It will help debug this.

shubham-goel commented 2 years ago

Actually, I think the scene scaling works but the silhouette loss weight is ill-suited for objects that occupy a very small part of the image. I think that centre-cropping the images and changing the focal length (accordingly) during data-loading will fix the issue. I can add this to the data-loading pipeline if you share your data with me.

Thanks again!