Projected textures incorrectly placed on provided mesh

[UsaidPro]

Describe the problem I am trying to texture a video game map using photogrammetry. The photogrammetry correctly creates a textured model by itself. However, when I try to provide a more-accurate in-game version of the map as a mesh to the Texturing node the final result shown in Blender has some of the textures "scrambled" and put in different positions than expected. When exporting the mesh with correct scale/orientation from Blender, I did follow the settings listed here in the wiki (including preserving the vertex order) when saving the .obj in Blender and tried modifying the "fill holes" and "padding" settings under Texturing in Meshroom to no avail.

Screenshots Here is a screenshot of a portion of the map. Clearly, some textures are correctly assigned to their locations. However, some are not. I marked with a red rectangle where one texture has shifted from on top of the object to underneath.

Above:

Below:

This is not necessarily a case of "wrong side" textures - sometimes the correct texture is on a face far away from where it is supposed to be.

Desktop (please complete the following and other pertinent information):

• OS: Windows 10
• Python version: 3.7.4
• Qt/PySide version: Whatever came with Meshroom 2020.1.1, looks like Qt5.3
• Meshroom version: 2020.1.1 Release version
  • Binary version (if applicable): 2020.1.1

Additional context I tried using various Meshroom-to-Blender imports from directly importing the .obj in Blender, using meshroom2blender, and using Blender-Addon-Photogrammety-Importer. Regardless of what I used, it was always the same output.

[natowi]

Do do you have the issue when loading the obj in Meshlab?

[UsaidPro]

Same issue when importing the mesh directly into Meshlab. The black sections have shifted here because I reran the texturing with UDIM off and it moved the black sections around (still has the issue though).

[UsaidPro]

Is there a way to work around this issue by importing everything but the last texturing step into Meshlab? I am thinking I could just let Meshlab try applying the textures. If Meshlab can't do it then I can assume it is something wrong with my model.

The weird thing is, the textures are there. They are just below the object. Often the bottom of the object is better-textured than the top although all the cameras/textures were on the top part.

[natowi]

With Meshlab I just wanted to check if there was a problem with the texture or the blender import.

when I try to provide a more-accurate in-game version of the map

Did you extract the 3d model from the game and want to use to texture it? Maybe your model is slightly offset, resulting in the texturing mismatch? Similar effects can occur, when the mesh is heavily modified or changed in its original orientation.

Not sure why you want to use the workflow to begin with, there are probably better ways to get the textured model.

[UsaidPro]

I did extract the 3d model from the game. I orientated the model with what the initial photogrammetry meshing created. I would think the model is slightly offset, but the fact many textures regardless of location are correct and I can find the correct textures for the black parts in seemingly-random positions indicates it is not an alignment issue no?

A better way of getting the textured model would be extracting the in-game textures. In fact, many of the in-game textures can be extracted. But not all of them - right now people are manually recreating these missing textures at a pretty slow rate, and the project I intend to do requires as accurate textures as possible. I figured photogrammetry might be an easier way to get the texture mapping, especially since I can save the effort of the model filtering since I have the actual in-game model.

I am right now rerunning the photogrammetry process on Meshroom 2021.1.0. Hopefully something has changed and this issue is fixed in the latest release.

[UsaidPro]

Using the 2021.1.0 release, I got an error when Texturing and providing my mesh. I created issue #1442 since this problem did not happen in the 2020.1.1 release although it might be related. Using mesh recreated in Meshroom worked fine, so it might be a problem with my mesh - but I do not know what it could be.

[natowi]

Are you sure you did not accidentally merge the reconstruction with the extracted Mesh when aligning both resulting in z-Fighting? Did you check the orientation of the normals of your imported model? Did you try another unwrapping mode? How many faces does your model have and what mode do you use? Do the "Force Visibility by all Vertices" or "Flip normals" result in changes?

There is also the option "Visibility Remapping Method" you could try (push):

The first method “Pull” works fine for a simple decimation, but starts to be problematic on a retopology with drastic simplification. The “Push” method iterates over each vertex of the dense reconstruction mesh and tries to push the visibilities into the closest triangle in the low-polymesh (after retopology). This improves the final textures. The default mode “PullPush” combines both approaches.

[UsaidPro]

I did not merge the reconstruction with the extracted mesh. I checked the orientation of the normals. The normals all looked correct, and there was no difference in normal orientation between the incorrectly-textured portions and the correctly-textured ones. I did not try another unwrapping mode. This is because the model has 852k faces which is large enough that judging from the instructions only Basic unwrapping with >600k faces would work.

I have tried using "Force Visibility by all Vertices" in the 2021 version which errors out. I will rerun in the 2020 version tonight and see if it does something. I will also try changing the remapping method.

I tried "Flip Normals" in the 2020 version and it did not result in any change whatsoever.

[UsaidPro]

~Using "Force Visibility by all Vertices" on the 2020.1.1 version results in the same issue as #1442 . Turns out that specific parameter does not like the mesh I provide. What could be a potential reason for this?~ Nevermind, for some reason now the texturing in 2020.1.1 also mysteriously errors out regardless of parameter. This only happened after I installed 2021.1.0 - does downloading/running the Meshroom binaries from Alicevision update the system path somehow?

[natowi]

does downloading/running the Meshroom binaries from Alicevision update the system path somehow?

No, but sometimes there are remnants in the cache folder if you made many changes and load the same project. It can confuse the node. Use the clear cache on the node and try to run it again.

If nothing works, you could still use blender (and a Meshroom blender plugin to read the .mg or .sfm) to reproject the images on your mesh using the known camera positions to project from. Similar to https://youtu.be/8NYNiayHvJI?t=375

I´m out of ideas on your issue and don´t have the data or time (sorry) to reproduce your problem.

[UsaidPro]

It is alright. Thank you for the link! Since the camera positions are accurate in Meshroom, I guess doing the texturing in Blender is feasible and probably would map onto it better. Thank you for your help!

[natowi]

Glad I could help a little. Here you can find the plugin I was referring to: https://github.com/SBCV/Blender-Addon-Photogrammetry-Importer As you can see in the example you only need to project the images on the mesh similar to the process described by the video posted above. Good luck :)

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[stale[bot]]

This issue is closed due to inactivity. Feel free to re-open if new information is available.