Texture swapping problem

[loveaca-sunlight]

Hi, thank you so much for providing such an amazing job!

I want to ask some questions about Texture swapping.

If I want to customize the content of the texture swapping, how should I do point annotating in Blender and export the data? Handy to provide the blender script you use?

Does thetexture_swapping_dtu_scan63.json file contain all the information needed to perform texture swapping? Sorry, I didn't understand the full flow of texture swapping from reading the code. So, I'm also curious about the respective meanings of the parameters in the json file (eg "T_s_m" and "corr").

[chobao]

Hi, loveaca-sunlight! Following our paper, there are 3 steps to customize the region of the texture swapping.

1. Annotate the region you want to swap on each mesh with blender. Our code recognizes the swapping region by the vertex color. All white nodes are selected as swapped candidates. The step can be done with the vertex paint mode in Blender. Please note that the vertices order of exported mesh should be the same as the original mesh. After that, the path of the main mesh(main_mask_mesh) and affiliated mesh should be written on the config. https://github.com/zju3dv/NeuMesh/blob/86ebce817aeba64a00473e05d9d5ad413dbe4147/render_texture_swapping.py#L276
2. Align the swapped region in Euclidean space. T_s_m denotes the similarity transformation from the main mesh(painted mesh) to the affiliated mesh(provide texture). corr denotes the still points when deforming mesh with as rigid as possible. The step can be easily done by enabling estimate_srt and our method will compute T_s_m, corr automatically with your intervention https://github.com/zju3dv/NeuMesh/blob/86ebce817aeba64a00473e05d9d5ad413dbe4147/render_texture_swapping.py#L435
3. Run the code and enjoy it.

[loveaca-sunlight]

Thanks a lot, I've made some progress with your tips.

I'd also like to double-check that your mention of "the vertices order should be the same" means that I shouldn't delete any vertices in Blender? If I paint one of them black and the other white in vertex drawing, shouldn't it affect the vertex order?

[chobao]

Hi, the full code has been released! Please take a look at it for more details.

• In our implementation, the vertices should not be deleted. However, I think some tricks can be employed to index the vertex correctly before and after deletion.
• Painting the vertex will not affect the vertex order if you check the keep the vertex order when exporting mesh in Blender.

[chobao]

Close the inactive issues. Feel free to reopen it if the problem still needs to be solved.