about adding texture

[YaqiChang]

ECON outputs only have front-view color, if I provide consistent backview 2d image of the model, how can I apply it on . Acutually, I find the code first find the front view of model, then apply vertex_color from source image I don't know whtether my understanind is correct or not?

            if cfg.bni.texture_src == 'image':
                final_colors = query_color(
                    # verts, gace, image
                    torch.tensor(final_mesh.vertices).float(),
                    torch.tensor(final_mesh.faces).long(),

                    in_tensor["image"][idx:idx + 1],     #idx
                    device=device,
                )
                final_mesh.visual.vertex_colors = final_colors
                final_mesh.export(final_path)

[YuliangXiu]

Correct, the vertices are categorized as either "visible" or "invisible," as shown in the query_color function. If you possess a back image, you can attempt to assign the colors of its pixels to the invisible vertices.

This approach is not flawless because some vertices may still be obscured in both the front and back views. Therefore, in order to achieve a genuinely full texture, blending must be performed.

[YaqiChang]

thanks a lot !!

[YuliangXiu]

If an appropriately aligned back image is available, it is possible to replace the existing back normal map (which was predicted from the front image using back-branch of normal estimator) with a more detailed normal map (predicted from the back image using front-branch of normal estimator). This substitution can significantly improve the quality of the reconstruction.

If works, a pull request would be appreciated. 😃

[yuedajiong]

I have implemented: fill backward texture from backward image. may I submit these code?

000_0_full.obj.zip

[YaqiChang]

I have implemented: fill backward texture from backward image. may I submit these code?

000_0_full.obj.zip

I just can't add textire to obj at same time, because the color will be overwrite by the normal color , could you please share the code. By the way, my way to get consistent backward image is to generate image is: using the 2d projection of the ECON output obj, and use canny+depth model with stable diffsuion.

HERE IS EXAMPLE

[tomorrow1238]

I have implemented: fill backward texture from backward image. may I submit these code? 000_0_full.obj.zip

I just can't add textire to obj at same time, because the color will be overwrite by the normal color , could you please share the code. By the way, my way to get consistent backward image is to generate image is: using the 2d projection of the ECON output obj, and use canny+depth model with stable diffsuion.

HERE IS EXAMPLE

May I ask if the method used here to obtain consistent backward images is an open source tool on the internet. Could you please share it with me

[YaqiChang]

I have implemented: fill backward texture from backward image. may I submit these code? 000_0_full.obj.zip

I just can't add textire to obj at same time, because the color will be overwrite by the normal color , could you please share the code. By the way, my way to get consistent backward image is to generate image is: using the 2d projection of the ECON output obj, and use canny+depth model with stable diffsuion. HERE IS EXAMPLE

May I ask if the method used here to obtain consistent backward images is an open source tool on the internet. Could you please share it with me

I just write a gradio demo which can let the reconstruction and texture genrateation work in just one web page. The main tworkflow is to get the backend projection of recon obj, then use this picture as the input of controlnet.

[YuliangXiu]

@ChangYaQi @yuedajiong

We recently released a new work, TeCH. Given a single image, TeCH could produce a full textured avatar with both intricate geometric details and consistent high-quality texture.

Homepage: https://huangyangyi.github.io/TeCH/ Code: https://github.com/huangyangyi/TeCH

[alexwang2013]

I have implemented: fill backward texture from backward image. may I submit these code?

000_0_full.obj.zip

@yuedajiong Hi, could you please share the code to implement the following function?

If an appropriately aligned back image is available, it is possible to replace the existing back normal map (which was predicted from the front image using back-branch of normal estimator) with a more detailed normal map (predicted from the back image using front-branch of normal estimator). This substitution can significantly improve the quality of the reconstruction.