search

vt-vl-lab / 3d-photo-inpainting

[CVPR 2020] 3D Photography using Context-aware Layered Depth Inpainting
https://shihmengli.github.io/3D-Photo-Inpainting/
Other
6.9k stars 1.11k forks source link

Edge map not being used in colour inpainting #105

Open violetamenendez opened 3 years ago

violetamenendez commented 3 years ago

Hi,

I have just found the following line in the code, which is setting the resize_rgb_dict['edge'] to 0, which is the edge map that is being inputted into the colour inpainting network. Is this a typo? Was this set to 0 on purpose to see what the effect of not using an edge map was? Or am I missing where the actual edge information is going into the network?

Thank you for your time.

https://github.com/vt-vl-lab/3d-photo-inpainting/blob/60ce4fcc5f8dc37a2b65bb72ef6287addc024bbf/mesh.py#L1694

violetamenendez commented 3 years ago

Reading the supplementing material of the paper I just found the following line:

"The input edge values in the synthesis region are similarly set to zeros for depth and color inpainting models, but remain intact for the edge inpainting network."

I feel like I am missing something, as my understanding is that you are inpainting the edge information in the synthesis area with the edge inpainting network, so you can then condition the output of the depth and colour inpainting networks on these edges. So wouldn't it be the other way around, the edge values are set to 0 in the edge inpainting network, so the network tries to recreate the missing information, but once they are inpainted they remain intact for the other networks?

Thank you for your time :-)

violetamenendez commented 3 years ago

Hi,

If you ever get around to looking into this. I would like to know more about your training process and how you generate the edge information to train the depth and colour models.

Thank you!

pandadreamer commented 3 years ago

I'm also confused about why setting the edge value to 0