Closed wangzheallen closed 3 years ago
Sorry, line 173-210 is a little dirty. I did not explain it clearly.
These codes are used to alleviate the problem in the resampling of vertices. Let me describe the problem first:
When resample the vertex location from a UV map, bilinear interpolation is utilized. This may cause a problem on the vertices near the edges of UV parts.
As shown in the figure below, the red and green vertex are interpolated using the 4 grid pixels. However, 3 of them will have zero value because there are outside of any triangle, which will cause artifacts on the resampled mesh.
In the codes of densbody-pytorch, they dilate the UV map for a pixel when generating the UV map. However, we found that dilation may cause some new artifacts when the UV map resolution is not high enough. It may mix multiple UV parts.
In addition, our uv_generator assumes every pixel to be related with only 3 vertices. So we are not able to use the pre-calculated dilation weight. And we can only realize the dilation process after the location map is generated, which is space-consuming.
So we use another way to solve this problem. When generating the location map, for any grid pixel that is used in the resampling process, if it has zero value, we assign it with the value of the vertex near to this pixel. For example: the 4 grid pixels are used in the resampling process of the red vertex, but 3 of them are with zero value. So these 3 pixels should be assigned with the value of the red vertex.
Some pixels may be related with multiple vertices. For example: the 4 grid pixels are related with both red and green vertex. In this condition, we decide the pixel value according to the interpolation weight.
Thanks for the kind clarification! This solves my concern! Will the operation of 'replacing the grid value with nearest vertex' results in non-smooth/erratic mesh? as suggested by https://gitee.com/seuvcl/CVPR2020-OOH/issues/I1PALL?from=project-issue
In our experiments, this may leads to a few erratic vertices in low resolution UV map. If high resolution is used, such as 256×256, both dilation and our process is okay.
Thanks for the answer!
Hi Wang, I am trying to understand the bary-weight calculation process. Basically you are trying to mapping the uv space faces to xyz space faces and want to solve the weight between them. However, I start to get lost from https://github.com/zengwang430521/DecoMR/blob/master/models/uv_generator.py#L179 and fail to understand why and what is the theory/intuition behind that about the calculation of w, u, v?
I know from https://github.com/zengwang430521/DecoMR/blob/master/models/uv_generator.py#L205 you are trying to construct a mask in UV space, but the lines before that seems confusing to me.