Building MM and Debugging ModDBInterop

[Frosferes2]

I've spent some time familiarising myself with VS and F#. If I can get MM to build successfully, I can move on to debugging the issues I brought up earlier.

I'm using VS Community 2022. When I attempt to open and build the MMDotNet solution, I get notifications that I'm missing specific versions of the .NETFramework reference assemblies (4.0, 4.5 etc.). But when I try to install these, the installer says they're already part of my OS and won't proceed. The reference manager says the targetted assemblies were not found.

Is there a way to solve the missing references using Paket or the Github build options you mentioned before? Apologies for the elementary questions. Once this is solved there shouldn't be any more barriers to debugging.

On a side note, I've made a Blender script to generate a akeleton for imported MM meshes. Bone positions are approximated using the weighted mean position of all the vertices in a given vertex group, where mean weights come from the vertex weights. If you are interested, I can share it with you, though I'm not really sure how to do so via Github. I also want to make a partner script that can use this skeleton to symmetrize the vertex groups for more convenient weight painting.

[jmquigs]

I use vs community 2019, the github action uses this as well. Not surprised that 2022 has Reference errors but I always struggle to fix them.

If you follow the github action you should be able to get it to build https://github.com/jmquigs/ModelMod/blob/master/.github/workflows/dotnet-desktop.yml

You can ignore the TPLib for now, that lib isn't needed (its just binaries of d3dx and similar files that I didn't want to host in a public repo).

[Frosferes2]

Okay I figured out the problem. I had to install the required .NET targetting packs. But since the .NET 4.0, 4.5 etc. are out of support for VS 2022, I had to add VS2019 to the installer and add only the targetting pack components. The app is building and running correctly now.

I'm going to try to implement proper tangent and binormal calculation using the reference site in the script comment to see if this fixes the light deflection issues.

[Frosferes2]

I realised the same thing you did in #10, i.e. that we can export the tangent space directly from blender instead of calculating it ourselves. I have written my own extraction method which ensures every loop normal/tangent/binormal has a unique vertex index, and are then also sorted by those same indices. This should ensure each vertex is assigned the correct normal, tangent and binormal.

Now we have tn and bn lines in addition to vn lines, which should be able to be imported by MM. This is something I will work on another day.

Since I have edited the export script, this branch will now conflict with the other one I made a pull request for. Should I merge the two branches when I have finished the edits to the MM code?

[jmquigs]

That's cool! I'm interested in seeing if those blender-generated vectors work better. I was disappointed with the results I got from the DXMesh library, but I couldn't be sure if it was just because I had a "dirty mesh" with duplicate verts or faces and something like that.

You can go ahead and merge it into your existing PR if that is the easiest. It could just be a new feature of moving to the more recent version of blender. FWIW your original PR will be hard to merge directly, because from what I can tell it doesn't seem backwards compatible, and it modifies the mmobj scripts in place (I would prefer to have separate script directories for both old and new versions of blender, so that I can continue to use 2.79b, since I have not yet mentally upgraded to the more recent versions.)

One thing I have strived for is to make sure the mmobj format is stable - for instance you don't need to reexport mmobj files when upgrading to a new version of MM. So any changes to the structure of the mmobj need corresponding changes in the F# loader so that it is able to load both old and new versions.

This would similarly apply for this change. if the bn and tn lines are available, then fine use that data, otherwise the F# code should fall back to whatever it was doing before (even if that produces sub optimal results, its better than just straight failing to load a mod.)

[Frosferes2]

@jmquigs You're correct that the new scripts are not compatible with Blender 2.7x. I think it may be possible to have install.py recognise the Blender version you're using and install the appropriate scripts, but it's a question of if it is worth trying to implement this. The code changes I made to the normals writing protocol should be cross compatible, they would just need copying over. How you want to proceed here is up to you. I feel the majority of potential users will be on the latest Blender version, so maybe making the 2.7x scripts optional is best? Though, without modifying install.py, they would probably have to be installed manually. That, or they could be contained in their own individual nested zip files.

I am just making some changes to the F# code. I have added tangent and binormal resize arrays and log lines to readOBJ(). Is the mesh object from this function passed directly to modmBinormalTangent()? Also, if I'm not mistaken, it appears only the tangent is being passed to the renderer, and the binormal is not present in this function. Is the binormal computed internally by the renderer from the normal cross tangent? I am still getting familiar with F#, so apologies for the frequent questions. I will make the old tangent calculation a fallback for if the tangent/binormal data is missing from the mmobj.

Also, when I build the app and try to launch GW2 I get a popup with the error Start failed: Can't find source files for copy. What is the best way to fix this? Do I need to change the build location?

EDIT: After adding the bin with the x64 DX dlls to the repo, the game now launches, but MM does not load correctly. The log outputs this: ThreadId(1)/36329ms: Error creating CLR: CLRInitFailed("Error: Managed code version mismatch. Ensure that the d3d9.dll loaded by the game is the latest version. You may need to copy the new version in from your ModelMod directory. Click 'Start' in the ModelMod Launcher for more details.") The DX dlls in the GW2 directory are the same as the ones in the repo bin. If I launch MM from the exe in the release build, this error doesn't happen. Any idea what's causing this?

[Frosferes2]

I'm trying to figure out what components are missing from the build version that prevents it from working like the release. The setup instructions in the dev guide appear to be outdated as there is no longer a solution named ModelMod in the project. Unfortunately Im struggling to troubleshoot this myself due to my inexperience with VS.

[jmquigs]

@Frosferes2 You can get a "managed code version mismatch" if the F# code does not match the version the rust code expects. The managed code version is here: https://github.com/jmquigs/ModelMod/blob/49f01fd5cc7139526769d789b1bb637ba6c0dd46/MMManaged/Interop.fs#L60C37-L60C37 While the rust code version is here: https://github.com/jmquigs/ModelMod/blob/49f01fd5cc7139526769d789b1bb637ba6c0dd46/Native/dnclr/src/dnclr.rs#L85C10-L85C10

Both are 3 in the committed code. Since I presume you are not rebuilding the rust stuff, I would expect that to be "3". You can try printing the versions that are being compared in the F# code before it throws the exception to see what rust is passing in.

Its important for these versions to to be the same, because if they aren't the rust code won't have the proper layout of the F# interop interfaces, which can lead to mysterious crashes.

[Frosferes2]

@jmquigs Initialising logging early in Interop and printing the native versions for the Rust and F# code shows a discrepancy as you said: the Rust code is version 2 and the F# is 3. I am using the d3d11.dll from the MM Beta 1.2.0.66, which I assume the native version is read from when the CLR is initialised. The compile date of the rust code reported by the log file aligns with when this release was published: ModelMod initialized, built with rustc: 1.68.0 2023-03-06, git hash: be4246665daedbf6f07aab3582513ecf30230506, build date: 2023-03-24 14:18:05.456161900 +00:00. The fact that the packaged build works by itself leads me to believe the native version is 2 in the build but not the repo for some reason. Am I wrong here? It's a little odd.

Removing the fail condition allows the CLR to initialise regardless, and the MM UI loads in game. But I assume this is not best practice, since you said version differences can lead to unexpected errors. Do you know what the differences are between native Rust versions 2 and 3? If I were to build the version 3 Rust binaries myself, how would I go about doing it?

[Frosferes2]

@jmquigs I figured out how to build the Rust binaries and moved and renamed the newly generated hook_core.dll to the MM bin. The build is now fully working and I have been able to start debugging.

Exciting news! I've discovered the source of the lighting bug in the F# code and it is not related to the tangents after all. When WriteElement is called, both the vertex index and vertex normal index are extracted from the element index record v. For some reason I am not sure of, the vertex normal indices are out of sync with the actual vertex indices in this record. I wrote both these values to the log and found that the vertex normal index values would not exceed around 300 and would often repeat, despite the mesh containing several thousand vertices. This would explain why all the normals face a similar direction: they are all copies of only the first few vertex normals.

I changed both modmNormal and modmBinormalTangent to call the normal and tangent vectors respectively using the vertex index instead of the normal index and this seems to almost completely fix the lighting. There is still some weirdness, but this might be due to my texture or it could be related to the axis basis of the vectors as I mentioned in a past comment. I say this because it seems to me like the lighting is correct, but oriented in the wrong direction. I will have to test this further.

This method will cause an error if the length of the vertex list does not exactly match the normals list, and I still want to implement the pre-calculated tangent space from Blender. It will take some more F# learning to implement this intelligently though.

[Frosferes2]

@jmquigs Sorry for the incessant comments. I just wanted to let you know that I believe I've managed to fix the issues with blend weight mode. Changing line 466 from UNorm to UInt allows the mode to load. However, after doing this I got the same bug as I mentioned in this comment from nearly 2 years ago.

After some investigation, it turns out the problem was with writeMeshBW. Occasionally when adding a new vertex to the mesh, its weights will be rounded in the byte array such that the sum of the array is not 255. When this happens, the vertex does not have a valid binding and stretches off in world space.

To correct this, we can add the difference between 255 and the array sum to the highest (first) weight value. The sum of a byte array is also a byte, and so will overflow if over 255. To solve this we can initialize the weight array as ints, and then convert to byte when passing to the binary writer. This seems to have resolved the issue as none of my remodels have has this stretching problem since implementing this.

I would like to commit the changes I've made to a new PR soon. The changes to the managed code can exist independently of the Python stuff, as I'm not sure how you would like to handle the Blender scripts. I'm still not too familiar with Git, so I don't want to flood the page with tiny PRs. When you can, please let me know how you want to go about this.

[Frosferes2]

@jmquigs Hi again. I just discovered that the position transformations that MM uses to translate between the coordinate systems for OGL and DX is incorrect. At present, going from Blender to DX gives MM the transformations: rot_y_180 rot_x_90. This results in transformations that are oriented correctly, but mirrored in the X axis. This can be easilly missed, as if the mesh is symmetrical in the X axis, then there is no noticable difference in terms of the mesh geometry.

Blender's coordinate system is right handed, whereas DX is left handed. There is therefore no way to map one onto the other without one axis being mirrored. If this were applied to the current transformation list, it would look like: flip_x rot_y_180 rot_x_90. However, a more direct translation list would be flip_z rot_x_90. According to the MS website, to translate a right handed coordinate system into a left handed one, we need to flip the sign on one of the axes, and reverse the winding on the mesh triangles by swapping the second and third vertex index of each triangle. Implementing a position flip is easy enough, but I am still working on the reverse winding.

This also means that the snapshot transformations are incorrect. Which expedites the need to have working snapshotting for DX11. Problem is, I really don't know what to do to fix this. So I would need some leads from you if I were to attempt it.

[jmquigs]

Hey @Frosferes2 sorry I've been not responding, my life over the past few weeks has been much more complicated than I'd like.

I've actually known about that inverted X for quite a while now but never fixed it. I think what be involved is for the snapshot code to come up with a new transform convention name and export the verts using that fixed convention. For backwards compatibility we'd still want to be able to load the old transform though (but they could continue to use the unfixed axis). Additional on a re-export the python code could optionally fix the export to use the new transform. I'd least that is how it would look in a perfect world.

The DX11 snapshotting code is actually sort of working as far as I recall though its been months since I looked at it. The issue is that with DX11 the game needs to be started with a special flag so that snapshottable data is copied into system memory. This is enabled with a "precopy" flag as seen here: https://github.com/jmquigs/ModelMod/blob/master/Native/hook_core/src/hook_device_d3d11.rs#L697 . Its ok to run the game in snapshot mode all the time, as it garbage collects data after a period of time, its just a bit of a performance drain, especially when the game is loading a lot of assets.

I had started to modify the launcher so that it could be started in either precopy or "playback" mode but never finished that work. I got stuck on what kind of design would be optimal to pass the flag to the rust code, a silly thing in retrospect. Its still pretty unlikely that I'll be able to get to it soon. But if you wanted to play around with snapshotting yourself you'd need to start there. You may need to build the rust code as I'm not sure if the beta version I made had the latest changes to enable that.

[Frosferes2]

Hi @jmquigs don't worry about not responding, life takes priority after all. I hope things settle down for you soon.

I have added a new 'flip' keyword to the 3D transform parser, similar to what is used in the 2D UV transform. This instructs the transformer to take the negative value of whatever vector element is being flipped. After doing this, changing an existing Blender mesh's transform strings from rot_x_90 rot_y_180 scale_0.1' to 'flip_z rot_x_-90 scale_0.1 is able to import the mesh such that it's chirality is the same between the game and Blender. However, since the Blender mesh is now mirrored compared to what it should be, it needs to be corrected by scaling all vertices along the X axis by -1 in edit mode, making sure the transform is centered on the object origin. I think the new transform basis should work for snapshotting as well once it's going. Perhaps this new basis could be added to the snapshot profiles list. Bare in mind that this basis affects the normals/tangents as well as vertex positions, so position mirroring will likely mean the lighting is also inverted.

On that note, I have been able to get the lighting on the imported mesh accurate enough that I can't tell if any further discrepancies are because of the transform basis or something to do with the mesh export step. This is something I will look into when snapshotting is going. I have been able to build the Rust code with Cargo locally by fudging the Git hash. I'll have a go at setting this precopy flag manually and see what happens. I have aimed to ensure the changes I made to the F# code are backwards compatible with previous mods. I take it you're not going to be available to review them at the moment given IRL commitments, right? Perhaps I will push my changes onto my MM branch and submit a pull request once everything is done. We can discuss individual code changes is said PR later, hopefully when you have more time.

[Frosferes2]

@jmquigs indeed you were right. Forcing GLOBAL_STATE.run_conf.precopy_data to true enables snapshotting again. I had to build the release version of the Rust native, as the debug version causes GW2 to crash. Is this a known issue?

Some initial observations about the snapshotting:

1. A position axis flip in the snapshot profile works to extract a mesh with the correct mirroring, but this also causes the face normals to be flipped. This is to be expected, but for some reason doesn't happen when reading a mmobj back into MM. I may need to set up a face winding reversal function to only call when snapshotting and not loading mods.
2. The snapshotter outputs several texture files per object: tex_0, tex_1 etc. Texture 0 is the diffuse map combined with the transparency layer, and opens as intended. However, some other texture types appear to be corrupted or perhaps incorrectly formatted, as they will not open in the image editors I tried. These maps could include normal, specular, metalic (in the case of armor) etc. I would like to be able to access these maps as well. I wonder if you know what the problem with them may be?
3. Certain components of some objects do not appear to snapshot correctly. For example, I am trying to snapshot the Dark Tyrant outfit, and some of the plate parts including the gauntlets and the chestplate are not appearing in the snapshot. Do you know of why only some of the outfit would be captured like this? Other parts like the hanging cloth in this outfit and several back items are also not captured. I assume this has something to do with the cloth physics these pieces have.
4. Snapshots don't seem to be generating the VBDecl file that MM wants to create a new mod. What does this file contain that is necessary for mods?

[jmquigs]

Hey @Frosferes2 , glad you got it sort of working

I always build in release build. If it crashes in debug, well that may or may not be a problem. Possibly the rust code is panicking, which would crash the whole game. I never use debug because its too slow. For iterative dev builds I use this script under git bash to copy the dll and run the game: https://github.com/jmquigs/ModelMod/blob/master/Native/hook_core/rb.sh

On other issues:

1. I am not sure what is going on here
2. Yes I noticed this as well. Could be that the format is a newer dx11 specific format and tools don't know how to load it. It could also be that the d3dx save texture function doesn't know how to to interpret the data and is filling the file with garbage. d3dx is very old and very obsolete. Probably MM should use DirectXTex or some newer library to copy out those images. Or it could be that the data is specific to the shader being used at render time, in which case no tool would be able to preview it. The code to save textures is here, as you can see it just calls out to d3dx: https://github.com/jmquigs/ModelMod/blob/master/Native/d3dx/src/d3dx.rs#L203
3. Yes this is a long standing issue. The GW engines are odd in that they will render certain complex meshes both with GPU-based animation (which MM can modify) and software animation (which MM can't use) And it will use both on a single object at the same time. Software rendered pieces will be fully transformed into their world position and animation frame. If you load them up in blender (you can directly import them from the snapshot dir, you don't need to create a mod), you can click on them in the object explorer and use the snap->object to origin feature to view them. These cannot be modified using modelmod, though I believe you can hide them with a deletion mod.
4. VBDecl used to store the d3d9 vertex layout. In d3d11, a file containing vertex elements should be written out instead (though I can't remember if I did this or not in the the snapshot code). I also don't think I modified the mod creation tool to support that new file. Blender doesn't need it, but it is required when the mod is loaded, so the mod yaml file should reference it.

Of course feel free to fork and branch so that you can commit your work. I won't be offended, in fact I'm happy someone is finally working on it other than me. The project certainly has some limitations but its still good for some use cases, I'd like to see it continue to work with gw2 at least.

[Frosferes2]

@jmquigs I've been rather enjoying working on this project. It's been a great way to learn about lower level languages and the IDE/Compiler/Git workflow. I'm just wondering what the theoretical limits of MM are. Maybe if I look into the Rust side of the code more I will learn about how the renderer works and what is/isn't possible

Face normal direction is based on the order of vertices in the triangle. When you invert an axis in a mesh, you change its handedness. To translate a triangle made for a left handed coordinate basis to a right handed one, you also have to reverse the order of vertices in the triangle, otherwise the triangle will be facing the wrong direction. I've already implemented this, so don't worry too much about it.

I will look at the mesh and texture stuff later. Out of curiosity, when you say some geometry is inaccessible by MM, does that mean inaccessible with the current implementation or is it just completely impossible? If I modify a mesh which has flowing cloth on part of it, like the Dark Tyrant outfit, will it lose this component or is it rendered seperately?

I assume the _VB file generated by the snapshot is the DX11 equivelant of _VBDecl? Are you aware of any documentation of this sort of file and what needs to be done to port the mod loader over? Knowing where the launcher checks for this file when creating a mod would also be useful.

[Frosferes2]

@jmquigs Hi again, hope you enjoyed the holiday season! I was poking around the screenshotted textures to try and figure out what the issues with them were. Turns out that most of the textures aren't actually broken, they were simply in a format that most image editors couldn't parse. Visual Studio has its own image editor that is capable of handling the more arcane DDS formats and was able to open almost all of the textures that were extracted. These can then be saved as some lossless format like png for external editing.

Inspecting the textures in VS shows that Texture0 is the combined diffuse/transparency maps, and Texture1 is the normal map with no alpha channel. Something I noticed with the normal maps was that they were an odd color compared to the light blue neutral that is typical of this kind of map: Google says that this can be caused by an inverted face normal sampling direction when baking the map. Changing the snapshot profile seems to have no effect on this. Regardless, I have been able to modify one of these normal maps and add a Tex1Path argument to my mod config. I was pleasantly surprised to see that it seemed to work in-game without issue. Whether or not it's worth the effort of fliping the snapshot normal sample direction, assuming this is doable, remains to be seen.

Most other maps I've seen appear to be misc color maps or some other kind of data I'm unsure of in various formats. The one exception being Texture2, which sometimes appears when snapshotting certain objects. This texture is significantly larger than others attached to the same object (16MB vs 1MB) and could not be opened by the VS image editor. Inspecting this texture with the VS hex editor shows that the entire file is filled with empty data (which translates to blank white pixels), except for a couple of lines near the beginning which state the image format. The file also ends on an incomplete line, which seems to indicate that the generation process was abruptly cut off. Perhaps like you say, d3dx is unable to interpret the data in this texture slot and just generates junk. If I were to guess what this map is supposed to be, it could be metallic or specular data, as it seems to generate when snapshotting armor with reflective components, but I can't say for sure.

[jmquigs]

Hi @Frosferes2 , cool that most of the textures appear to be valid at least in visual studio. That's a good sign that even the old d3dx is snapping them right, it just doesn't know how to interpret them. Perhaps there are some transforms that could be applied on export (and then again on load) to change them in and out of a format that blender and other tools can understand. Unfortunately I'm very weak on texture stuff. Some people working on the reshade project might be able to help more, since I presume they run into this stuff all the time with what they do.

I agree that green normal map is very unusual I've never seen one like that before.

I agree it sound like the 16MB texture does have an format that is unrecognized. It might be a previous backbuffer itself (which could be used for some reflection effects), or, if the format is totally unknown even the size could be invalid.

While I' here let me answer some questions from your earlier post:

I will look at the mesh and texture stuff later. Out of curiosity, when you say some geometry is inaccessible by MM, does that mean inaccessible with the current implementation or is it just completely impossible? If I modify a mesh which has flowing cloth on part of it, like the Dark Tyrant outfit, will it lose this component or is it rendered seperately?

Usually the data is present, but because its fully transformed into world space, including its exact animation frame, it can't really be modded, because this violates an assumption MM makes, which is that the base mesh is in the unanimated rest pose. So a mod can be swapped out with that and the shader will animate with no problems. Its a bit hard to explain but, if you tried to mod a particular frame of the cloth, what you'd likely find is on subsequent frames, the delta vectors between vert and mod positions would get so far out of whack that you'd have a polygon dance on the screen. The only way around this would be to copy that data back to the CPU on each and every frame, do a computation to figure out where the mesh is relative to some current mesh, apply a mod, and then copy it back to GPU. Which would be very difficult and slow in a game like GW2. Years ago I did experiment with this for GW1 (which makes much heavier use of CPU-animation), but that code is gone now and it never worked great to begin with.

I assume the _VB file generated by the snapshot is the DX11 equivelant of _VBDecl? Are you aware of any documentation of this sort of file and what needs to be done to port the mod loader over? Knowing where the launcher checks for this file when creating a mod would also be useful.

The only docs are the rust code and the snapshot code in F# sadly. But I believe its just a copy of the raw data structures that I read out of d3d11. But yes, the decl and layout contain the same type of information, basically the offset into each vertex structure of where things like positions, normal, textures can be found. One irritating thing about DX11 in GW2 is that it periodically just randomly change these (usually after a level swap) in a running game for meshes that were already loaded, it changes the format around (even though its still functionally identical) and MM doesn't notice. This can cause mods to stop appearing that were once working in that session. A full reload sometimes fixes this as it can force MM to update its cached formats.

[Frosferes2]

@jmquigs I agree it would be great to be able to talk with some of the Reshade guys about texture/lighting topics. There is still some weirdness related to the vertex normals that has an impact on the lighting. Even after translating the normals corectly into the DirectX coordinate basis, they still require some additional rotations to appear correctly. I wonder if the coordinate system used by game's rendering shader is somehow offset from the basis used by the world space. If this is the case, it may necessitate additional xform instructions in the mmobj file. The issue of the flipped normal maps is also something I would like to explore, along with some other strange texture types I have discovered in snapshotting.

It's difficult to know what to do with the VB files. All the VBDecl file I have seen have been the same across all my old mods and are very short. By contrast, the VB files are very variable in length, which is dependent on the size of the extracted mesh. I assume this data is raw binary and can't be opened in some other app? I suppose if I want to work on this I will have to dive more into the snapshotting code to understand the data structure more. Given that MM is now mostly doing what I want it to, I haven't had as much personal motivation to investigate this. I have been able to create new mods by substituting the missing VBDecl file with one from my previous mods, but this is obviously a jerryrig solution.

One other thing is that I have developed a few Blender scripts to help with creating MM mods. These aren't part of a full addon, but could be incorporated into one if desired. Let me know if you find these interesting:

I have one which generates an armature for an imported mesh by estimating bone positions based on the weighted mean positions of the vertices in each group. This allows for easier weight painting, along with the other utilities of having a vertex deforming armature to work with. The bones are unstructured, as I can think of no way to estimate what the bone heirarchies should look like from the available data, but this script automates the most important and tedious parts.

Creating mods for different armor sets can be challenging because various faces are culled from the body mesh whenever a piece is equipped, and the resulting body mod has variable vertex group index ordering, making it difficult to use the same Blender mesh for both mods. This is something which I think you mentioned as being a problem. I have developed a name profile system which stores different possible names for each bone in an armature as custom property strings, which can be freely swapped between using a script. When the armature a mesh is skinned to changes the name of its bones, the names of the vertex deforming groups in the mesh are updated automatically, which is very handy.

A new armature can be generated from an imported reference mesh which has been culled by armor. I made another script which can match the bones in this reference armature to the bones in the armature used by the main body based on the relative distances between the reference and target bones. It then adds the names of these reference bones to the target bones as a name profile. This means the same body can be used for multiple mods by swapping between name profiles. The necessary faces can then be culled using a mask modifier. I was thinking of a similar script to use the name profile system for bone symmetrisation as well.