nmoehrle / mvs-texturing

Algorithm to texture 3D reconstructions from multi-view stereo images
Other
931 stars 328 forks source link

how can i fix this optimization error in global_seam_leveling function #170

Closed kumo-park closed 3 years ago

kumo-park commented 3 years ago

hello. nmoehrle i am using your great code for texturing. but i got stuck in some problem.

the thing i am trying to do is to make unseen faces textured with vertex color.

for example

Screen Shot 2021-05-28 at 8 53 52 AM

picture 1. this is unseen faces

i try to give a patch to unseen faces in picture 1. so i did with below steps in generate_texture_patches function a. create blank image for an unseen face (mve::FloatImage::Ptr image = mve::FloatImage::create(imgW,imgH,imgC)) b. fill the image with vertex color with interpolation algorithm c. extract patch from that image d. register that patch into a patch list ( TexturePatch::Ptr texture_patch= TexturePatch::create(texture_views->size()+1+i, faces, texcoords, image) texture_patches->push_back(texture_patch) ) e. make a VertexProjectionInfo and then push the info into a list ( VertexProjectionInfo info = {texture_patch_id, projection, {face_id}} vertex_projection_infos->at(vertex_id).push_back(info) ) f. iterate steps from a~e for each unseen face

after these step i got a result like below picture

Screen Shot 2021-05-28 at 8 54 24 AM

picture2. this looks what i expected

but sometime the result is changed in global_seam_leveling function like this

Screen Shot 2021-05-28 at 8 53 00 AM

picture 3. wrong result from global_seam_leveling function

it's like an alien movie. so i want you to give me a good intuition to fix this problem. thank you~

nmoehrle commented 3 years ago

How much padding do your artificial patches have? I could imagine that this is an out of bounds access when sampling the edges because the texture coordinates are too close to the border of the artificial atlas patches for interpolation.

kumo-park commented 3 years ago

How much padding do your artificial patches have? I could imagine that this is an out of bounds access when sampling the edges because the texture coordinates are too close to the border of the artificial atlas patches for interpolation.

so the solution you are thinking is to add some space with padding in my artificial patches right? i think if i add some extra padding in my artificial patches, the atlas map is getting more bigger. is there any solution to make atlas map compact? i mean after making my artificial patches, the atlas map is made like picture 1 below. so i want to make compact atlas maps like picture 2(it is for original patches). picture 2 looks more natural

result_material0015_map_Kd picture 1 . weird atlas map including artificial patches.

result_material0000_map_Kd picture 2. good atlas map for original patches