Vertex Groups are not importing

[Hetaneko]

I am trying to mod PGR. The character that i am trying to import to blender has vertex groups but the plugin is not importing them. I dont know if it's because it has same offsets.

element[12]: SemanticName: BLENDWEIGHT SemanticIndex: 0 Format: R8G8B8A8_UNORM InputSlot: 0 AlignedByteOffset: 0 InputSlotClass: per-vertex InstanceDataStepRate: 0

element[13]: SemanticName: BLENDINDICES SemanticIndex: 0 Format: R8G8B8A8_UNORM InputSlot: 0 AlignedByteOffset: 0 InputSlotClass: per-vertex InstanceDataStepRate: 0

FrameAnalysis.zip

[DarkStarSword]

Yes, the fact that AlignedByteOffset is reused from POSITION means the plugin will assume they are invalid and skip them.

You can also see that there isn't room for them in the buffer - vb0 stride is 40 bytes per vertex, which is entirely filled up by the position, normal and tangent: Bytes 0-11 is POSITION, a float3 (4 byte float x 3 components = 12 bytes) Bytes 12-23 is NORMAL, a float3 (4 byte float x 3 components = 12 bytes) Bytes 24-39 is TANGENT, a float4 (4 byte float x 4 components = 16 bytes)

These kind of invalid semantics turn up fairly often, and I'm not entirely sure why - my best guess is that these are quirks of how certain engines handle unused semantics. In the past I have confirmed that they definitely are reusing the existing data by adding debugging to the shader to confirm the values that the shader receives in these semantics match the data in the other semantics using the same offset, but usually the shaders aren't actually using the values, or they might be using them to e.g. read the input position via a TEXCOORD instead of SV_Position. The later wouldn't make sense in this case though, since the BLENDINDICES / BLENDWEIGHT is defined as a UNORM, while the POSITION is a FLOAT - totally different encodings.

See if you can find the skinning vertex shader that goes with these and post it here (use assembly if possible, 3DMigoto tends to break skinning shaders when dumped as HLSL - I've got some fixes for that scheduled for the 1.3.17 release). That might give us a clue as to where the shader is really getting the vertex weights from (e.g. some engines like Unreal are known to pass certain semantics in structured buffers bound to a texture slot instead of the vertex buffers), then maybe we can do something with that.

[Hetaneko]

I dont know if these can help you. Shader.zip

[DarkStarSword]

I meant export the shader with 3DMigoto. In d3dx.ini, set:

hunting = 1

marking_actions = asm

verbose_overlay = 1

(you can include additional marking actions if desired. hunting can alternatively be set to 2 and soft enabled at runtime via numpad 0)

Reload d3dx.ini (F10) or restart game, and play to where the shader is in use.

Cycle vertex shaders with numpad 4/5 until the relevant vertex shader (9f0cb0b114c65c16 based on your frame analysis filenames) is selected

Press numpad 6 to dump the shader into the ShaderFixes folder

If you don't have a numpad you can change the key bindings in the d3dx.ini

[Hetaneko]

I am sorry for misunderstanding. I will do it after the game is out of maintenance.

[Hetaneko]

I get a different hash when the character is skipped. I guess i have posted the wrong files.

0b0d109072cd6c57-vs.txt

FrameAnalysis-2023-09-26-191444.zip

[DarkStarSword]

I guess i have posted the wrong files

Not necessarily - Unity games tend to draw every object twice using two different sets of shaders, so you probably just grabbed it from the other shader before. When you cycle through the shaders you get a different appearance when each shader is selected (IIRC one makes it disappear, the other removes the colour or something) - after a while you get used to picking which is which while hunting, and switching to the pink hunting mode might help as well. For what we are looking for I don't think it matters which set you grab (the vertex shaders of both should calculate the same position on screen).

So I don't see any sign that this shader is applying skinning - the output position o0 is being calculated by multiplying the input position v0 by a 4x4 matrix in cb1[0 through 3] (probably a model / local to world matrix) followed by another 4x4 matrix in cb2[17 through 20] (probably a view-projection matrix):

dcl_output_siv o0.xyzw, position
...
mul r0.xyzw, v0.yyyy, cb1[1].xyzw
mad r0.xyzw, cb1[0].xyzw, v0.xxxx, r0.xyzw
mad r0.xyzw, cb1[2].xyzw, v0.zzzz, r0.xyzw
mad r0.xyzw, cb1[3].xyzw, v0.wwww, r0.xyzw
mul r1.xyzw, r0.yyyy, cb2[18].xyzw
mad r1.xyzw, cb2[17].xyzw, r0.xxxx, r1.xyzw
mad r1.xyzw, cb2[19].xyzw, r0.zzzz, r1.xyzw
mad r1.xyzw, cb2[20].xyzw, r0.wwww, r1.xyzw
mov o0.xyzw, r1.xyzw

The rest of the shader we can ignore as we are only interested in how the output vertex position in o0 is calculated.

If there was skinning being applied I would expect to see an array of matrices being indexed, e.g. something like this (from DOAXVV), note here that cb2 (which the reflection info left in this shader indicates was named "skinning_matrix_block") is being indexed with a variable register r2.w rather than a constant:

  loop
    ...
    mad r4.xyz, cb2[r2.w + 0].xyzx, r4.wwww, r4.xyzx
    mad r5.xyz, cb2[r2.w + 1].xyzx, r4.wwww, r5.xyzx
    mad r6.xyz, cb2[r2.w + 2].xyzx, r4.wwww, r6.xyzx
    ...
  endloop

I haven't had a look at exactly how Unity applies skinning before - I'll take a look at a couple of Unity games I own and see if it does anything unusual (such as calculating the skinning in a prior pass). In the meantime, can you confirm for me that this model we are looking at definitely is rendered in different poses by the game, and is not just a static model?

It's also possible that there might be two different versions of this model used at different times in the game, one with skinning when it needs to be posed and the other without when it doesn't - if that's the case, we want to make sure we are looking at the version with skinning.

[DarkStarSword]

Also can you just confirm for me which version of Unity this is? Right click on the exe -> properties -> details -> Product version (or File version)

[Hetaneko]

I just import the matching ib.txt to the blender: Then i export as vb/ib back without any changes:

The weird thing is the character was not moving at all before the maintenance. Now i am getting this black silhouette.

.ini file:

[TextureOverride1] hash = 55c962be vb0 = ResourceCostumeReplaceGirlVB vb1 = ResourceCostumeReplaceGirlVB2 ib = ResourceCostumeReplaceGirlIB drawindexed = auto handling = skip

[ResourceCostumeReplaceGirlVB] type = Buffer stride = 40 filename = untitled.vb0

[ResourceCostumeReplaceGirlVB2] type = Buffer stride = 76 filename = untitled.vb1

[ResourceCostumeReplaceGirlIB] type = Buffer format = R16_UINT filename = untitled.ib

Also can you just confirm for me which version of Unity this is? Right click on the exe -> properties -> details -> Product version (or File version)

2018.4.30.36255

Edit: I imported the whole dump to blender and there was not a single vertex group

[DarkStarSword]

Alright, I've had a chance to look at what Unity is doing, and it is indeed calculating the skinning in a previous step. Just in case the game you are looking at is somehow different to the one I'm looking at, I'll detail how I came to this conclusion so you can verify the same thing is happening on yours. I'm using Dreamfall Chapters to check this, and grabbed a frame analysis dump from here this scene, basically just placed the camera so that almost nothing other than Zoë and a single wall was in the camera frustum to keep the frame analysis as minimal as possible:

The text in that screenshot is showing the contents of vb0 for Zoë's body (vb0=ff8cdd6a identified through the typical hunting workflow) using my constant/other buffer live view debug shader, and it revealed that the values were changing each frame. For reference I configured the debug shader as follows:

[Include]
include = ShaderFixes\debug_cb.ini

[ShaderOverrideBodyVS]
hash = f38ceff2ebef6c80 
checktextureoverride = vb0

[TextureOverrideBody]
hash = ba0d3373
Resource\debug_cb\Buf = vb0

I also dumped the vertex shader corresponding to that draw call (vs=b2a69a4319b6cbd4), and noticed that it is quite similar (not identical) to the one you found, in that it does not appear to apply any skinning. Like yours, importing the vertex buffer from these draw calls into Blender does not import any blend indices or weights (though I should note a point of difference to your samples - in my case the input layout does not even list any BLENDINDICES or BLENDWEIGHTS at all, neither valid nor invalid). It also appears to have a pose already applied, which wasn't really evident in your samples and part of the reason I'm detailing all this so you can try to verify if your case is the same, or if there might be another difference we need to investigate further:

All of these facts leads me to believe that the skinning has already been applied to the positions in the vertex buffer at some prior point. To track that down, the next step is to look at the frame analysis log.txt file. Firstly I locate where the vertex buffer is bound to the pipeline for the relevant draw calls - in my case the body is drawn in multiple draw calls (7, 8, 9, 10, 21, 39, 40, 41, 42, 43) for different materials (skin vs clothes) and render passes - I choose draw call 7 somewhat arbitrarily and locate the entry for draw call 7 showing where the buffer with hash ba0d3373 is bound to vertex buffer slot 0 via IASetVertexBuffers:

000007 IASetVertexBuffers(StartSlot:0, NumBuffers:1, ppVertexBuffers:0x000000000C73E5B8, pStrides:0x000000000C73E5B0, pOffsets:0x000000000C73E5C8)
       0: resource=0x0000000034E2CA70 hash=ba0d3373

It's worth keeping in mind that some other engines (apparently not Unity) might optimise their API calls so that the vertex buffer will only be bound once (e.g. in draw call 7) and continue to be used in subsequent draw calls (8, 9, 10), so if you can't find an IASetVertexBuffers call for the draw call and vertex buffer you are looking at, you may need to look backwards to find the most recent call that set the vertex buffer slot you are looking for. From my log.txt, it appears that this version of Unity lacks such an optimisation and re-binds every vertex buffer individually for every single draw call, so we don't need to worry about that in this case.

Now we search backwards in the file to see where this buffer was last written to. I would generally recommend searching for the memory address (in the above example 0x0000000034E2CA70) rather than the hash, as it is possible for there to be multiple buffers sharing the same hash (though in this case there is not so searching for either will work). This leads us to find this line in draw call 1:

000001 SOSetTargets(NumBuffers:1, ppSOTargets:0x000000000C73E480, pOffsets:0x000000000C73E588)
       0: resource=0x0000000034E2CA70 hash=ba0d3373

SO is "Stream Output", a mechanism in DirectX that allows for a special geometry shader to write vertex data back to a buffer for use in future draw calls - this sounds exactly like what we are looking for. Note that 3DMigoto only has limited support for stream output geometry shaders (e.g. you might notice that no GS are shown in the overlay in most Unity games, despite the GSSetShader line for this draw call clearly indicating that a geometry shader is in use), but I don't think that limitation will impact us here since we shouldn't need to modify that geometry shader. 3DMigoto does have support for assigning buffers to stream output slots, however this support has never been tested and could well turn out to have bugs - that may impact us if we find we need to replace the buffer that passes the skinned mesh to the later draw calls (e.g. if we find the buffer is too small we may need to replace it with a larger one), but let's cross that bridge when we come to it.

Anyway, let's quickly note down some of the other info the log file has about this draw call in case we need it later:

000001 VSSetShader(pVertexShader:0x00000000354B2008, ppClassInstances:0x0000000000000000, NumClassInstances:0) hash=22d300f58071ea9f
000001 GSSetShader(pShader:0x00000000354B3F88, ppClassInstances:0x0000000000000000, NumClassInstances:0)
...
000001 IASetVertexBuffers(StartSlot:0, NumBuffers:2, ppVertexBuffers:0x000000000C73E4C8, pStrides:0x000000000C73E578, pOffsets:0x000000000C73E570)
       0: resource=0x000000000B3BBCB0 hash=1c626101
       1: resource=0x0000000034E27270 hash=bcf742a6

Here we have the hash for this vertex shader (22d300f58071ea9f) that we can use if we need to add a [ShaderOverride] section for it later. There is no pixel shader (the PSSetShader line passes null (all zeroes) for pShader), and as noted above there is a geometry shader, but since it's a stream output variant 3DMigoto has not hashed it like it would a regular geometry shader. We also have the hashes of the vertex buffers that it is taking as input - if our theory is correct these should hold the unskinned neutral pose mesh with vertex weights we are looking for.

But, there's several problems trying to import these vertex buffers into Blender - there's no index buffer in this draw call, and the vertex buffers are point lists rather than triangle lists. I'm mulling over how I might be able to modify the script to handle this more easily, but I think for now our best bet will be to fudge the frame analysis dump a little to make the plugin happy. Exporting this back to the game might also be challenging, but I think that will be a problem to solve another day - for now, let's just get it into Blender.

FIrstly, I noticed the sample you sent only included the txt versions of the buffers in the frame analysis dump. I think it might be a good idea to redo that dump with buf enabled as well (e.g. analyse_options = dump_vb dump_ib txt buf), as we might want the full index buffers available (the .txt versions only include a subset of triangles that were actually used in each draw call, while the buf versions are the entire buffer and may include triangles from other draw calls as well) - I'm not yet positive if we will actually need or want that in the end, but it's at least good to have the option available until we have a finalised workflow for cases like these.

So, first step in fudging the frame analysis dump is to copy all the files for the stream out draw call (1 in my case) into a separate folder - otherwise the script might try to pull in data from other draw calls with the pre-skinned mesh.

Next copy one of the index buffers from the draw call we were looking at earlier (e.g. 7 in my case) into this folder and rename it so the plugin will use it for the stream out draw call, so in my case I copy the files:

000007-ib=63058da3-vs=f38ceff2ebef6c80-ps=d3d5d117ded2a3a8.txt
000007-ib=63058da3-vs=f38ceff2ebef6c80-ps=d3d5d117ded2a3a8.buf

and copy + rename them to pretend they came from the stream out draw call. Note that in addition to changing the draw call number I also have to replace the vs= and ps= to match the other files from draw call 1:

000001-ib=63058da3-vs=22d300f58071ea9f.txt
000001-ib=63058da3-vs=22d300f58071ea9f.buf

The next step is to edit both vertex buffer txt files for this draw call, and change topology: pointlist to topology: trianglelist near the top. Now, blender should be able to import the files with vertex weights and a neutral pose:

As I mentioned above, I'm only seeing the specific geometry that was drawn in draw call 7 here. To get the entire mesh loaded in we could either copy the index buffers from the other draw calls associated with the mesh and fudge things so they use the same vertex buffers as the stream out draw call (i.e. so that the "Auto-load related meshes" option will work when importing and you will get each piece nicely split up like the usual workflow we use in DOAXVV and other games), or enable the "Load .buf files instead" so that the entire index buffer is loaded into a single mesh (which may be better if our goal is to edit the vertex cloud before skinning gets applied since that isn't cut up at that point):

As I mentioned above, there are still some additional challenges we may need to solve in order to get an edited mesh injected back into the game, but at least we have got this far :)

[HazzyDevil]

I've attached a sample frame dump from Burning Blood which has the same issue here if you wanted to test it out on a non-unity game.

In terms of injecting the edited mesh back into the game, the way that the folks in the Genshin Impact modding server have gone about doing it is after importing the mesh into blender, they do a typical export using your blender script. They run a separate python script that splits the vb file from the blender export back into the separate buffer files which I assume would be done by taking into account the original stride and offsets for each bit of data and the .fmt file that gets created from your script which has the total stride. Just my assumption based on their bit of code here:

while i < len(data):
            import binascii
            position += data[i:i+48]
            blend += data[i+48:i+80]
            texcoord += data[i+80:i+stride]
            i += stride

    return position, blend, texcoord

The only issue with their script is the offsets are hardcoded so if the stride/offset for something is not as it appears above, then the script wont work, which does occur for some of the meshes. Another issue which is in both Genshin Impact and OPBB is that the mod when injected back into the game, for some reason will have portion of the mesh that gets drawn into the center of the screen like in the image below:

The fix here is to simply add a TextureOverride of the VB hash into the ini file (called TextureOverrideTestVertexLimitRaise in the sample below). Not too sure why that is the fix but it works. The explanation from SilentNightSound from the GI modding discord is because your blender script is actually exporting "around 10% more vertices than there was originally". And they aren't being drawn for some reason unless you add that random TextureOverride with the VB hash in it.

Here's a sample of the auto-generated ini file using their script which I've tweaked to also work on Burning Blood with no issues. I'll also attach the sample mod for Burning Blood that has the the buf files and the .ini below. The mod itself doesn't change anything to the mesh, but is just a test to see if it injects back into the game without issue, which it does! I also did a test that simply removed a portion of the mesh and injected that back into the game, and that works too. So fair to say it is possible.

; Test

; Overrides -------------------------

[TextureOverrideTestPosition]
hash = eb5b95a7
vb0 = ResourceTestPosition

[TextureOverrideTestBlend]
hash = 9f28cfda
vb1 = ResourceTestBlend
handling = skip
draw = 1457,0 

[TextureOverrideTestTexcoord]
hash = 12f8f9f5
vb1 = ResourceTestTexcoord

[TextureOverrideTestVertexLimitRaise]
hash = 5d28ebed

[TextureOverrideTestIB]
hash = 1383d74e
handling = skip
drawindexed = auto

[TextureOverrideTest]
hash = 1383d74e
match_first_index = 0
ib = ResourceTestIB

; Resources -------------------------

[ResourceTestPosition]
type = Buffer
stride = 48
filename = TestPosition.buf

[ResourceTestBlend]
type = Buffer
stride = 32
filename = TestBlend.buf

[ResourceTestTexcoord]
type = Buffer
stride = 16
filename = TestTexcoord.buf

[ResourceTestIB]
type = Buffer
format = DXGI_FORMAT_R16_UINT
filename = Test.ib

OPBB Frame Analysis.zip TestMod.zip

[DarkStarSword]

I've pushed up some updates to the script to add experimental support to import and export the pointlist topologies (and related dependencies like loading vertex data without an index buffer) that these engines are sending to the skinning shaders directly, HOWEVER I DO NOT ACTUALLY RECOMMEND WORKING WITH POINT LISTS. Nevertheless, there may be some utility for being able to open these, if nothing else we can at least see the uniform shape of the objects passed to these draw calls:

A big reason not to use them, besides the obvious fact that they lack triangles / faces, is that the normals and tangents are lost during this process (in Blender these are stored/calculated in the vertex-per-face data, so without faces we lose this), so the models will lose lighting if you attempt to export these back to the game:

Original:

Normals lost after importing + exporting through Blender:

If we do find a genuine need for working with point lists I can potentially try to resolve that by storing the normals + tangents in a vertex layer to at least preserve the ones that were already on the mesh, but I think it's going to be better to work with triangle lists if at all possible where we can manipulate custom normals and re-calculate the tangents on export.

@HazzyDevil thanks - I've added code so the script can load the point lists from the One Punch sample as well - they have (trivial) index buffers assigned for those calls that Unity doesn't use, but otherwise it looks to be doing much the same thing.

I'll have to take a closer look at what the Genshin modders came up with. It sounds like it's pretty similar to what I was thinking, though the details might be a bit different.

The way I'm kind of envisioning that it might work is that the script will search through the frame analysis log file to locate any vertex buffers that were previously used in an SOSetTargets call, and either load those instead of / merged with / or as additional vertex buffers to the ones from the later call (and if the buffers aren't present in the dump it could suggest the d3dx.ini changes necessary to capture them) - essentially automating the way I fudged the buffers in my previous post.

On export it can then provide a complete vertex buffer of the entire mesh to replace the inputs to the skinning pass (or possibly inject additional skinning calls which would not require the exported buffer to be complete, not yet sure which is the better option), as well as the vertex and index buffers to replace in the later draw calls that are drawing the individual pieces of the mesh.

[DarkStarSword]

In terms of injecting the edited mesh back into the game, the way that the folks in the Genshin Impact modding server have gone about doing it is after importing the mesh into blender, they do a typical export using your blender script. They run a separate python script that splits the vb file from the blender export back into the separate buffer files which I assume would be done by taking into account the original stride and offsets for each bit of data and the .fmt file that gets created from your script which has the total stride. Just my assumption based on their bit of code here:

I believe the version of my script they are using predates the support for multiple vertex buffers (which isn't surprising as I only added that relatively recently) - this looks more like a workaround to combine the vertex buffers together and split them apart so they could be used with the old version of the script, though it could well also be combining and splitting the pre-skinned vertex buffer as well.

The only issue with their script is the offsets are hardcoded so if the stride/offset for something is not as it appears above, then the script wont work, which does occur for some of the meshes.

Hopefully this is something the new multi vertex buffer support makes obsolete since it doesn't need these offsets hard coded, but I can't say whether this is something they would want to spend the time to integrate into their tools if they already have a mostly working solution.

Another issue which is in both Genshin Impact and OPBB is that the mod when injected back into the game, for some reason will have portion of the mesh that gets drawn into the center of the screen like in the image below:

The fix here is to simply add a TextureOverride of the VB hash into the ini file (called TextureOverrideTestVertexLimitRaise in the sample below). Not too sure why that is the fix but it works. The explanation from SilentNightSound from the GI modding discord is because your blender script is actually exporting "around 10% more vertices than there was originally". And they aren't being drawn for some reason unless you add that random TextureOverride with the VB hash in it.

So, on the one hand it's not surprising that they would need to increase the number of vertices in the skinning draw call (and I'll look at adding a draw=auto to the next version of 3DMigoto to simplify this), because even if Blender didn't add extra vertices a modder might, but I'm not quite sure how [TextureOverrideTestVertexLimitRaise] would be working from the excerpt you posted - that will change a couple of code paths within 3DMigoto, but I don't see an obvious reason it would fix anything. Either I'm missing something else in their ini files that explains it, or I'll need to make sure they aren't inadvertently exploiting some kind of bug that just happens to be working in their favor at the moment. They are shipping a custom version of 3DMigoto right? I guess it's also possible they just added a hack in that to e.g. increase the size of the stream output buffer to accommodate the extra vertices.

[HazzyDevil]

Okay did some digging and yeah it appears that in addition to that TextureOverrideVertexLimit raise in the ini file, they edited the dll files. Not sure if it was all of them or not but if you wanted to compare them, you can find their 3Dmigoto instance below 3dmigoto GIMI (for playing mods).zip

In terms of the pointlist topology, the way the GI python script works is you tell it the vertex shader that handles the animation/posing of the mesh, and the vertex buffer hash of the mesh. The script will then look at the vertex count in the file using the VB hash, and then find a corresponding txt file with the weights using the vertex shader hash as that should also have the same vertex count. I guess the only issue here would be if it were to find a different mesh that had the same vertex count using the same vertex shader. But doing it this way, it by passes the need to import the mesh using pointlist topology as it's using the trianglist topology that's noted in the .txt/.bin files of the vertex buffer as opposed to the same files for the weights that are in pointlist.

[DarkStarSword]

@SilentNightSound From what I understand you have modified 3DMigoto for your GI importer - is the source code for your fork available anywhere, and is there anything you would like me to integrate from it for the next official 3DMigoto release?

[SilentNightSound]

Hello! It is great to finally have a chance to speak with you, I have been reading through your notes for a long time trying to figure out how the program works and to get it to function with Unity games like Genshin.

For the multiple VB issue, yes the scripts were written before the plugin was updated to support loading from multiple vertex buffers so I wrote a workaround that combines all the buffers into a single one and then modified the plugin to split them apart later when loading back into the game. For games like Genshin it was actually a bit more complicated because the game does the skinning calculations in a separate draw call from the one that actually draws the model to the screen, so the buffers need to be replaced across multiple calls in order to function properly (+several of the buffers are mislabled by the game and we needed to solve the pointlist issue you mentioned in your post, which is why I wrote a script that helps collect and sort the data into a readable format).

This also led to why I had to modify the dll slightly in order to support Genshin - I am not completely sure if it was my lack of knowledge, but I could not find a way to increase the size of the buffers after they had already been created. Since the skinning calculations are done in an earlier step and then fed back into the buffer, the original size is used instead of the updated size when you change the buffers leading to a portion of the model not being drawn.

This was especially problematic because whenever blender has multiple normals for a single point the plugin has to export duplicates to handle it (because if I am understanding correctly 3dmigoto can only handle having single normal per point) which led to the ~10% increase in size when exporting and thus part of the models wouldn't be drawn.

For games like PGR which were originally mentioned (among many other mobile ports), I have looked into them and it appears that the skinning is completely done prior to ever arriving at the buffers. Faces in genshin also suffer from this issue. I have attempted a few methods such as reversing the skinning calculations in hlsl and recalculating face keys, but have not had much success - those games seem to be limited to texture and shader edits only.

One other feature was I added the ability for the program to read back some data from the GPU into the CPU (https://stackoverflow.com/questions/17414491/how-to-read-vertices-from-vertex-buffer-in-direct3d11) since I was unable to find a way to get things like buffer values back into the ini files. This was specifically so we could make mods that change their status based on the current HP of characters (e.g. the clothing break mechanic from senran games)

The last difference is that at the genshin modding discord, we are trying to implement draw-independent transparency - genshin does not have a standardized order for how it draws objects, so we are running into problems with trying to blend transparent objects since they can be drawn before the object that would be behind them. In order to do this, we are exporting additional render targets which I believe requires a code change since we could not figure out how to do it from ini only.

Other than that, the only other thing of note was adding in the ability to load custom armatures and animations in the game (e.g. see https://streamable.com/9yyogz) but that was not a dll change and was done primarily through hlsl coding (and sadly never 100% completed since I couldn't find anyone to help me with the calculations and reached the limit of my skill - I was hoping to use this to create fully custom bones, but only managed to get half-way).

[DarkStarSword]

In terms of the pointlist topology, the way the GI python script works is you tell it the vertex shader that handles the animation/posing of the mesh, and the vertex buffer hash of the mesh. The script will then look at the vertex count in the file using the VB hash, and then find a corresponding txt file with the weights using the vertex shader hash as that should also have the same vertex count.

Yeah, that makes perfect sense.

I guess the only issue here would be if it were to find a different mesh that had the same vertex count using the same vertex shader.

Yes, and if we want the final solution to be able to apply more generally to other possible variants of this pattern we shouldn't even really make the assumption that the vertex buffer input to the skinning shader would even be the same size as the stream output buffer - in Unity's case it is because they split the vertex buffers up such that vb0 and so0 have the same position, normal + tangent semantics as each other, and the semantics that do differ between the draw calls (blendweights, texcoords, etc) are in vb1/vb2 instead. But there's nothing stopping other games from e.g. sticking everything in vb0 and omitting some things from so0.

I think my preferred approach will be to search the frame analysis log file to identify the corresponding unskinned buffers, I think that will end up being the most reliable approach to finding them.

But doing it this way, it by passes the need to import the mesh using pointlist topology as it's using the trianglist topology that's noted in the .txt/.bin files of the vertex buffer as opposed to the same files for the weights that are in pointlist.

Yes, I think that's definitely the way to go. In some ways adding support for importing point lists was more a 'why not, might be interesting' than it actually being something I thought would definitely be part of the final solution. I may still use it - depends on how exactly I end up handling vertex buffers from both draw calls.

[DarkStarSword]

Hello! It is great to finally have a chance to speak with you, I have been reading through your notes for a long time trying to figure out how the program works and to get it to function with Unity games like Genshin.

For the multiple VB issue, yes the scripts were written before the plugin was updated to support loading from multiple vertex buffers so I wrote a workaround that combines all the buffers into a single one and then modified the plugin to split them apart later when loading back into the game. For games like Genshin it was actually a bit more complicated because the game does the skinning calculations in a separate draw call from the one that actually draws the model to the screen, so the buffers need to be replaced across multiple calls in order to function properly (+several of the buffers are mislabled by the game and we needed to solve the pointlist issue you mentioned in your post, which is why I wrote a script that helps collect and sort the data into a readable format).

This also led to why I had to modify the dll slightly in order to support Genshin - I am not completely sure if it was my lack of knowledge, but I could not find a way to increase the size of the buffers after they had already been created. Since the skinning calculations are done in an earlier step and then fed back into the buffer, the original size is used instead of the updated size when you change the buffers leading to a portion of the model not being drawn.

Yes, changing buffers after the fact is not easy because 3DMigoto doesn't wrap the buffer passed to the game so it can't replace it later

3DMigoto can alter some parameters of textures at the time of creation (refer to override_resource_desc()), but it looks like we don't have an implementation of that for buffers - I might see about adding something to that for the next release.

Alternatively, it's usually possible to create a new buffer at runtime using the copy_desc keyword and some overrides to make a buffer that is based on an existing one from the game, but with some attributes like the size altered. You would then basically catch each time the game goes to use the original buffer and switch it out for the modified one. I've used this for other circumstances (offhand, my WATCH_DOGS2 mod substitutes the HUD render target to solve an SLI flicker bug, and Dreamfall Chapters uses this to replace a bunch of the buffers involved with light shafts with stereo versions). For the specific case we are looking at here though this would also involve replacing a buffer used as a Stream Output, and while I do have code to do that in 3DMigoto, it's never been tested and based on the crash I just got - may still need some work ;-)

This may become easier in the geo-11 fork of 3DMigoto as that does wrap buffers + textures (as it needs to create stereo pairs of these) and therefore could potentially replace the wrapped buffer after the fact. I'm not sure if that's a use case that @davegl1234 (author of that fork) has considered, but it's certainly something we could look at doing in the future. However at present geo-11 is far less stable than 3DMigoto and has a lot of bugs and regressions that impact this type of modding, so I wouldn't recommend using it if you don't need stereoscopic 3D support (at least not yet - someday I'd like to see it completely replace 3DMigoto).

This was especially problematic because whenever blender has multiple normals for a single point the plugin has to export duplicates to handle it (because if I am understanding correctly 3dmigoto can only handle having single normal per point) which led to the ~10% increase in size when exporting and thus part of the models wouldn't be drawn.

Yeah, that's right (well, it's not really a 3DMigoto limit as such, more a limit of the buffers we pass to DirectX)

For games like PGR which were originally mentioned (among many other mobile ports), I have looked into them and it appears that the skinning is completely done prior to ever arriving at the buffers. Faces in genshin also suffer from this issue.

You mean on the CPU? Yeah, that would definitely be a show stopper since 3DMigoto can't intervene there. You could potentially write a custom tool that hooks into the game to solve it... but at that point it becomes questionable whether it's worth sticking with runtime modifications like 3DMigoto or going the other approach of reverse engineering the game file formats and patching them offline, and potentially risky if it's a multiplayer title.

I have attempted a few methods such as reversing the skinning calculations in hlsl and recalculating face keys, but have not had much success - those games seem to be limited to texture and shader edits only.

I always meant to see if I could do something similar to that to create usable skeletons from the blend matrices, but I never quite got around to figuring out the maths or if it would even be possible. It would definitely be interesting to take another crack at this at some point :)

One other feature was I added the ability for the program to read back some data from the GPU into the CPU (https://stackoverflow.com/questions/17414491/how-to-read-vertices-from-vertex-buffer-in-direct3d11) since I was unable to find a way to get things like buffer values back into the ini files. This was specifically so we could make mods that change their status based on the current HP of characters (e.g. the clothing break mechanic from senran games)

I added something along those lines to 3DMigoto a while back to read out a value for use in the auto-convergence shaders I used in DOAXVV and some other games, but never got around to generalising it for use beyond that feature (and geo-11 now has built the auto-convergence feature in - @davegl1234 were there any changes to the GPU to CPU copying that might be relevant?). This sounds like it might be a worthwhile change to bring into the official 3DMigoto - is that something you might be able to polish up and send a pull request for (or alternatively @bo3b could potentially grant you commit access to the official repo)?

The last difference is that at the genshin modding discord, we are trying to implement draw-independent transparency - genshin does not have a standardized order for how it draws objects, so we are running into problems with trying to blend transparent objects since they can be drawn before the object that would be behind them. In order to do this, we are exporting additional render targets which I believe requires a code change since we could not figure out how to do it from ini only.

I think that should be possible from the ini - we can certainly create new render targets (copy_desc is useful to make a new target the same size as an existing one, with overrides in the [resource] section if you need it to use a different format or similar) and bind them to additional output slots on the pixel shader. If you need to change any of the other render state you can use [CustomShader] sections. Was there anything specific that was missing/difficult?

[SilentNightSound]

Yes, changing buffers after the fact is not easy because 3DMigoto doesn't wrap the buffer passed to the game so it can't replace it later

3DMigoto can alter some parameters of textures at the time of creation (refer to override_resource_desc()), but it looks like we don't have an implementation of that for buffers - I might see about adding something to that for the next release.

Alternatively, it's usually possible to create a new buffer at runtime using the copy_desc keyword and some overrides to make a buffer that is based on an existing one from the game, but with some attributes like the size altered. You would then basically catch each time the game goes to use the original buffer and switch it out for the modified one. I've used this for other circumstances (offhand, my WATCH_DOGS2 mod substitutes the HUD render target to solve an SLI flicker bug, and Dreamfall Chapters uses this to replace a bunch of the buffers involved with light shafts with stereo versions). For the specific case we are looking at here though this would also involve replacing a buffer used as a Stream Output, and while I do have code to do that in 3DMigoto, it's never been tested and based on the crash I just got - may still need some work ;-)

This may become easier in the geo-11 fork of 3DMigoto as that does wrap buffers + textures (as it needs to create stereo pairs of these) and therefore could potentially replace the wrapped buffer after the fact. I'm not sure if that's a use case that @davegl1234 (author of that fork) has considered, but it's certainly something we could look at doing in the future. However at present geo-11 is far less stable than 3DMigoto and has a lot of bugs and regressions that impact this type of modding, so I wouldn't recommend using it if you don't need stereoscopic 3D support (at least not yet - someday I'd like to see it completely replace 3DMigoto).

Yes, either updating the header or copying the data over to another buffer and changing the header are probably the best ways to do it I agree - back when I made the change ~1.5 years ago, I knew much less about how d3dx worked (or rather basically nothing at all about how it worked xD) so the only solution I was able to implement was to add a special command to the ini which would raise the size of the buffer at creation time. That method has the downside of wasting space when the extra size isn't needed though, and it results in a few glitches (such as the model needing to be unloaded/reloaded for the updated size to take effect, or not being able to dynamically set the limit).

For games like PGR which were originally mentioned (among many other mobile ports), I have looked into them and it appears that the skinning is completely done prior to ever arriving at the buffers. Faces in genshin also suffer from this issue.

You mean on the CPU? Yeah, that would definitely be a show stopper since 3DMigoto can't intervene there. You could potentially write a custom tool that hooks into the game to solve it... but at that point it becomes questionable whether it's worth sticking with runtime modifications like 3DMigoto or going the other approach of reverse engineering the game file formats and patching them offline, and potentially risky if it's a multiplayer title.

Yes, a lot of mobile ports and unity games do skinning in the CPU. In fact, I believe that is the default set by Unity and developers needed to upgrade to the pro version in order to use GPU skinning. Reversing the format wasn't an option for us sadly - while we know the file format, any attempts to change it for Genshin results in an immediate ban so we are limited to GPU changes.

I always meant to see if I could do something similar to that to create usable skeletons from the blend matrices, but I never quite got around to figuring out the maths or if it would even be possible. It would definitely be interesting to take another crack at this at some point :)

If you do decide to give it a try, let me know! I've made a decent amount of progress being able to create the skinning matrices for custom animations, and have a partial solution to solve out the matrix from an already posed model (though since each point can have up to 64 unknown variables, often there simply isn't enough data to solve out the equations). Implementing it in hlsl is another matter entirely though - my current solutions use the numerical python libraries, which isn't something that can be ported into shaders.

I added something along those lines to 3DMigoto a while back to read out a value for use in the auto-convergence shaders I used in DOAXVV and some other games, but never got around to generalising it for use beyond that feature (and geo-11 now has built the auto-convergence feature in - @davegl1234 were there any changes to the GPU to CPU copying that might be relevant?). This sounds like it might be a worthwhile change to bring into the official 3DMigoto - is that something you might be able to polish up and send a pull request for (or alternatively @bo3b could potentially grant you commit access to the official repo)?

Sure, I could try though I must admit my coding skills are not very strong - most of the changes I have made are more like hacks to get it working rather than proper fixes and features since even now I still lack a strong grasp of how exactly the d3dx pipeline works.

I think that should be possible from the ini - we can certainly create new render targets (copy_desc is useful to make a new target the same size as an existing one, with overrides in the [resource] section if you need it to use a different format or similar) and bind them to additional output slots on the pixel shader. If you need to change any of the other render state you can use [CustomShader] sections. Was there anything specific that was missing/difficult?

That is what I thought too - in fact, I feel like getting transparency to work should be much simpler based on previous messages I read on the DOAXVV modding forums. But for some reason the game seems to ignore any attempts to use custom textures to dictate transparency, and will only work if we specify blend factors (which can't do gradients and are limited to the entire part). My current workaround is to export multiple render targets containing transparent and opaque portions, then manually blend them at a later step (alternatively, if there was a way to do multiple passes by passing the output of a PS shader to another PS shader in the same draw call in sequence I could draw the opaque parts first followed by the transparent ones)

A big part of the problem is that the game draws characters before drawing the background, so even if we turn on transparency in OM it won't blend properly. And the game loves to shuffle the draw order between versions, so even if one version of the game draws body -> dress (which blends properly) the next might be dress -> body.

Any attempts to add additional render targets in ini or hlsl seem to be ignored - it just outputs the original set of render targets (can't change their parameters either - if an oX had only 1 channel, it remains with a single channel no matter what we do).

[DarkStarSword]

so the only solution I was able to implement was to add a special command to the ini which would raise the size of the buffer at creation time. That method has the downside of wasting space when the extra size isn't needed though, and it results in a few glitches (such as the model needing to be unloaded/reloaded for the updated size to take effect, or not being able to dynamically set the limit).

Honestly the extra space usage is usually going to be pretty negligible compared to the amount of VRAM on any modern GPU, and the only pressure would be from games that actively try to maximise their VRAM usage without going over, which I don't think is something Unity does, and even then if it does go over the driver can page resources on PC (unlike consoles where going over means an instant crash).

I gather that your hack is triggered from the TextureOverrideTestVertexLimitRaise name? I don't think we would want quite that hack in the official 3DMigoto (if it was a setting within the TextureOverride we could consider it, but probably not something based just on the name), but let's make sure we have some way to achieve this properly before 1.3.17 is released.

I assume the GI loader will continue using your fork where the hack will continue to work so existing mods don't get broken.

Yes, a lot of mobile ports and unity games do skinning in the CPU. In fact, I believe that is the default set by Unity and developers needed to upgrade to the pro version in order to use GPU skinning.

Oh, that's interesting... I should check that, I'm supposed to have a pro license through my day job (but I think it expired a few months back, I may have to pester my boss about renewing it... even though I'm not currently working on a Unity project).

Reversing the format wasn't an option for us sadly - while we know the file format, any attempts to change it for Genshin results in an immediate ban so we are limited to GPU changes.

Fair. Probably hooking into the skinning routine on the CPU would also be risky in that case, so doing everything through 3DMigoto / GPU makes the most sense. Reading memory might still be safe, most tamper detection systems wouldn't be able to detect that (there are ways they technically could, but I doubt any of them would actually go that far... then again, I don't know - I've never exactly gone in to work out exactly what they do and do not try to detect, because I don't really feel like dealing with them should be my battle and I usually just avoid any games that use them).

If you do decide to give it a try, let me know! I've made a decent amount of progress being able to create the skinning matrices for custom animations, and have a partial solution to solve out the matrix from an already posed model (though since each point can have up to 64 unknown variables, often there simply isn't enough data to solve out the equations). Implementing it in hlsl is another matter entirely though - my current solutions use the numerical python libraries, which isn't something that can be ported into shaders.

I'd certainly be interested in seeing what you've come up with in python so far :)

I added something along those lines to 3DMigoto a while back to read out a value for use in the auto-convergence shaders I used in DOAXVV and some other games, but never got around to generalising it for use beyond that feature (and geo-11 now has built the auto-convergence feature in - @davegl1234 were there any changes to the GPU to CPU copying that might be relevant?). This sounds like it might be a worthwhile change to bring into the official 3DMigoto - is that something you might be able to polish up and send a pull request for (or alternatively @bo3b could potentially grant you commit access to the official repo)?

Sure, I could try though I must admit my coding skills are not very strong - most of the changes I have made are more like hacks to get it working rather than proper fixes and features since even now I still lack a strong grasp of how exactly the d3dx pipeline works.

That's ok - we've had plenty of hacks added to 3DMigoto over the years to get something working in one specific game, and later expanded (or deprecated + replaced) the feature to be more generally useful once we have a fuller understanding of the problem better, or just as the command list grew and became more and more capable that we no longer needed a hack.

If you want you can send a pull request with your current implementation without worrying about polishing it too much, and @Bo3b and myself can review it and offer feedback/suggestions.

That is what I thought too - in fact, I feel like getting transparency to work should be much simpler based on previous messages I read on the DOAXVV modding forums. But for some reason the game seems to ignore any attempts to use custom textures to dictate transparency, and will only work if we specify blend factors (which can't do gradients and are limited to the entire part). My current workaround is to export multiple render targets containing transparent and opaque portions, then manually blend them at a later step (alternatively, if there was a way to do multiple passes by passing the output of a PS shader to another PS shader in the same draw call in sequence I could draw the opaque parts first followed by the transparent ones)

A big part of the problem is that the game draws characters before drawing the background, so even if we turn on transparency in OM it won't blend properly. And the game loves to shuffle the draw order between versions, so even if one version of the game draws body -> dress (which blends properly) the next might be dress -> body.

Yeah, it can get pretty complex. I'd need to take a deep look at what you've tried to work out why it wasn't working, and probably don't have time to dive that deep at the moment.

One thing I've thought might help for these cases is some kind of record + replay feature in 3DMigoto, since often we want to react to draw calls that were issued during an opaque render pass, but have our modified meshes / injected draw calls happen in a transparent render pass later in the frame. I don't have a particularly clear idea for how this would actually work, and so far we've usually been able to find ways to make things work without it - but it can be a real kludge.

Any attempts to add additional render targets in ini or hlsl seem to be ignored - it just outputs the original set of render targets (can't change their parameters either - if an oX had only 1 channel, it remains with a single channel no matter what we do).

This should work... I'm sure I've done this at some point - I've definitely done it in debugging (e.g. adding an extra ouptut to write some value I want to examine at some point in a pixel shader combined with debug_2d.ini to examine it live), though there might have been some gotchas that I've forgotten the details of now... I'm trying to think if I have done this in a released mod that we could refer to, but the only ones I can think of offhand (The Witness, Subnautica) were assigning a depth target, not a render target.

[HazzyDevil]

Decided to test the latest blender script and yeah it looks to import the drawn mesh (point topology), as well as the posed mesh (triangle topology) without issue for OPBB using the sample I posted above.

However, for some reason the vest doesn't look right. Something's off with the position of the vertices. It at least resembles the vest somewhat, but yeah not too sure what's up.

Vest 1: 000208-vb0=5d28ebed-vs=837d5f9733b757ee-ps=bd136ec315cf848d.txt Vest 2: 000045-vb0=5d28ebed-vs=70a701dd177cd067-ps=4f6da09bd0afde2a.txt

Strange that it's only the vest that looks off, and not any of the other meshes.

[DarkStarSword]

That could be down to a bad frame analysis dump - looking at the log file for draw call 45 I can see that it replaced vb1 and ib before the dump, and vb1 in particular wasn't backed up and restored (vb0 was). This is also evident in that the vb1 filename 000045-vb1-vs=70a70....txt lacks a hash after vb1, indicating that it was a buffer created by 3DMigoto rather than the game. Something might still be amiss here, but we should make sure we are looking at a clean dump with no replacements first.

It's generally a good idea to add a test on !frame_analysis in the ini files to prevent switching meshes when frame analysis is active to make sure the dumps will be clean. In DOAXVV I use this test (you can probably drop the $dump_modded_meshes part, that's there for situations where I specifically do want to see the results of replacing meshes in the dump):

if $costume_mods && (!frame_analysis || $dump_modded_meshes)

The log file also shows this matched both [ShaderOverrideLuffyClothes-Hat2-VS] and [ShaderOverrideLuffyClothes2-PS] causing it to run through [commandlistgeneral] twice, which might not be what you want.

000045 DrawIndexed(IndexCount:5208, StartIndexLocation:0, BaseVertexLocation:0)
000045 3DMigoto pre {
000045 3DMigoto  [ShaderOverrideLuffyClothes-Hat2-VS] run = commandlistgeneral
000045 3DMigoto  pre {
000045 3DMigoto   [commandlistgeneral] if $dump_skin: true {
000045 3DMigoto    [commandlistgeneral] analyse_options = deferred_ctx_accurate share_dupes dump_vb txt buf dump_ib dump_tex
000045 3DMigoto    [commandlistgeneral] dump = deferred_ctx_accurate vs-cb0
000045 3DMigoto Deferring Buffer dump: D:\Steam\steamapps\common\ONE PIECE BURNING BLOOD\FrameAnalysis-2023-10-02-230649\ctx-0x00000189E338C790\000045.0-[commandlistgeneral]-vs-cb0=7816b819.XXX
000045 3DMigoto   } endif
000045 3DMigoto   [commandlistgeneral] if $costume_mods: true {
000045 3DMigoto    [commandlistgeneral] checktextureoverride = ps-t0
000045 3DMigoto    [commandlistgeneral] checktextureoverride = ps-t1
000045 3DMigoto    [commandlistgeneral] checktextureoverride = ps-t2
000045 3DMigoto    [commandlistgeneral] checktextureoverride = ps-t3
000045 3DMigoto    [commandlistgeneral] checktextureoverride = ps-t4
000045 3DMigoto    [commandlistgeneral] checktextureoverride = ps-t5
000045 3DMigoto    [commandlistgeneral] checktextureoverride = ps-t6
000045 3DMigoto    [commandlistgeneral] checktextureoverride = ps-t7
000045 3DMigoto    [commandlistgeneral] checktextureoverride = ps-t8
000045 3DMigoto    [commandlistgeneral] checktextureoverride = ps-t9
000045 3DMigoto    [commandlistgeneral] checktextureoverride = ps-t10
000045 3DMigoto    [commandlistgeneral] checktextureoverride = ps-t11
000045 3DMigoto    [commandlistgeneral] checktextureoverride = ps-t12
000045 3DMigoto    [commandlistgeneral] checktextureoverride = ps-t13
000045 3DMigoto    [commandlistgeneral] checktextureoverride = ps-t14
000045 3DMigoto    [commandlistgeneral] resourcebakvb = ref vb0
000045 3DMigoto      copying by reference
000045 3DMigoto    [commandlistgeneral] resourcebakib = ref ib
000045 3DMigoto      copying by reference
000045 3DMigoto    [commandlistgeneral] checktextureoverride = vb0
000045 3DMigoto    [commandlistgeneral] checktextureoverride = vb1
000045 3DMigoto    pre {
000045 3DMigoto     [TextureOverride\Mods\TestMod\Test.ini\TestTexcoord] vb1 = resourcetesttexcoord
000045 3DMigoto       copying by reference
000045 3DMigoto    }
000045 3DMigoto    [commandlistgeneral] checktextureoverride = vb2
000045 3DMigoto    [commandlistgeneral] checktextureoverride = vb3
000045 3DMigoto    [commandlistgeneral] checktextureoverride = vb4
000045 3DMigoto    [commandlistgeneral] checktextureoverride = vb5
000045 3DMigoto    [commandlistgeneral] checktextureoverride = vb6
000045 3DMigoto    [commandlistgeneral] checktextureoverride = ib
000045 3DMigoto    pre {
000045 3DMigoto     [TextureOverride\Mods\TestMod\Test.ini\Test] ib = resourcetestib
000045 3DMigoto       copying by reference
000045 3DMigoto    }
000045 3DMigoto    pre {
000045 3DMigoto     [TextureOverride\Mods\TestMod\Test.ini\TestIB] handling = skip
000045 3DMigoto     [TextureOverride\Mods\TestMod\Test.ini\TestIB] drawindexed = auto -> DrawIndexed(5208, 0, 0)
000045 3DMigoto    }
000045 3DMigoto    [commandlistgeneral] vb0 = ref resourcebakvb
000045 3DMigoto      copying by reference
000045 3DMigoto    [commandlistgeneral] ib = ref resourcebakib
000045 3DMigoto      copying by reference
000045 3DMigoto   } endif
000045 3DMigoto  }
000045 3DMigoto }
000045 3DMigoto pre {
000045 3DMigoto  [ShaderOverrideLuffyClothes2-PS] run = commandlistgeneral
000045 3DMigoto  pre {
000045 3DMigoto   [commandlistgeneral] if $dump_skin: true {
000045 3DMigoto    [commandlistgeneral] analyse_options = deferred_ctx_accurate share_dupes dump_vb txt buf dump_ib dump_tex
000045 3DMigoto    [commandlistgeneral] dump = deferred_ctx_accurate vs-cb0
000045 3DMigoto Deferring Buffer dump: D:\Steam\steamapps\common\ONE PIECE BURNING BLOOD\FrameAnalysis-2023-10-02-230649\ctx-0x00000189E338C790\000045.1-[commandlistgeneral]-vs-cb0=7816b819.XXX
000045 3DMigoto   } endif
000045 3DMigoto   [commandlistgeneral] if $costume_mods: true {
000045 3DMigoto    [commandlistgeneral] checktextureoverride = ps-t0
000045 3DMigoto    [commandlistgeneral] checktextureoverride = ps-t1
000045 3DMigoto    [commandlistgeneral] checktextureoverride = ps-t2
000045 3DMigoto    [commandlistgeneral] checktextureoverride = ps-t3
000045 3DMigoto    [commandlistgeneral] checktextureoverride = ps-t4
000045 3DMigoto    [commandlistgeneral] checktextureoverride = ps-t5
000045 3DMigoto    [commandlistgeneral] checktextureoverride = ps-t6
000045 3DMigoto    [commandlistgeneral] checktextureoverride = ps-t7
000045 3DMigoto    [commandlistgeneral] checktextureoverride = ps-t8
000045 3DMigoto    [commandlistgeneral] checktextureoverride = ps-t9
000045 3DMigoto    [commandlistgeneral] checktextureoverride = ps-t10
000045 3DMigoto    [commandlistgeneral] checktextureoverride = ps-t11
000045 3DMigoto    [commandlistgeneral] checktextureoverride = ps-t12
000045 3DMigoto    [commandlistgeneral] checktextureoverride = ps-t13
000045 3DMigoto    [commandlistgeneral] checktextureoverride = ps-t14
000045 3DMigoto    [commandlistgeneral] resourcebakvb = ref vb0
000045 3DMigoto      copying by reference
000045 3DMigoto    [commandlistgeneral] resourcebakib = ref ib
000045 3DMigoto      copying by reference
000045 3DMigoto    [commandlistgeneral] checktextureoverride = vb0
000045 3DMigoto    [commandlistgeneral] checktextureoverride = vb1
000045 3DMigoto    [commandlistgeneral] checktextureoverride = vb2
000045 3DMigoto    [commandlistgeneral] checktextureoverride = vb3
000045 3DMigoto    [commandlistgeneral] checktextureoverride = vb4
000045 3DMigoto    [commandlistgeneral] checktextureoverride = vb5
000045 3DMigoto    [commandlistgeneral] checktextureoverride = vb6
000045 3DMigoto    [commandlistgeneral] checktextureoverride = ib
000045 3DMigoto    pre {
000045 3DMigoto     [TextureOverride\Mods\TestMod\Test.ini\Test] ib = resourcetestib
000045 3DMigoto       copying by reference
000045 3DMigoto    }
000045 3DMigoto    pre {
000045 3DMigoto     [TextureOverride\Mods\TestMod\Test.ini\TestIB] handling = skip
000045 3DMigoto     [TextureOverride\Mods\TestMod\Test.ini\TestIB] drawindexed = auto -> DrawIndexed(5208, 0, 0)
000045 3DMigoto    }
000045 3DMigoto    [commandlistgeneral] vb0 = ref resourcebakvb
000045 3DMigoto      copying by reference
000045 3DMigoto    [commandlistgeneral] ib = ref resourcebakib
000045 3DMigoto      copying by reference
000045 3DMigoto   } endif
000045 3DMigoto  }
000045 3DMigoto }
000045 3DMigoto analyse_options (one-shot): 00a00634
000045 3DMigoto Deferring Buffer dump: D:\Steam\steamapps\common\ONE PIECE BURNING BLOOD\FrameAnalysis-2023-10-02-230649\ctx-0x00000189E338C790\000045-ib=1383d74e-vs=70a701dd177cd067-ps=4f6da09bd0afde2a.XXX
000045 3DMigoto Deferring Buffer dump: D:\Steam\steamapps\common\ONE PIECE BURNING BLOOD\FrameAnalysis-2023-10-02-230649\ctx-0x00000189E338C790\000045-vb0=5d28ebed-vs=70a701dd177cd067-ps=4f6da09bd0afde2a.XXX
000045 3DMigoto Deferring Buffer dump: D:\Steam\steamapps\common\ONE PIECE BURNING BLOOD\FrameAnalysis-2023-10-02-230649\ctx-0x00000189E338C790\000045-vb1-vs=70a701dd177cd067-ps=4f6da09bd0afde2a.XXX
000045 3DMigoto Deferring Buffer dump: D:\Steam\steamapps\common\ONE PIECE BURNING BLOOD\FrameAnalysis-2023-10-02-230649\ctx-0x00000189E338C790\000045-vs-t0=5c2a48b3-vs=70a701dd177cd067-ps=4f6da09bd0afde2a.XXX
000045 3DMigoto Skipped 3DMigoto resource in slot vs-t120
000045 3DMigoto Deferring Texture2D dump: D:\Steam\steamapps\common\ONE PIECE BURNING BLOOD\FrameAnalysis-2023-10-02-230649\ctx-0x00000189E338C790\000045-ps-t0=d90f038d-vs=70a701dd177cd067-ps=4f6da09bd0afde2a.XXX
000045 3DMigoto Deferring Texture2D dump: D:\Steam\steamapps\common\ONE PIECE BURNING BLOOD\FrameAnalysis-2023-10-02-230649\ctx-0x00000189E338C790\000045-ps-t1=89099fd7-vs=70a701dd177cd067-ps=4f6da09bd0afde2a.XXX
000045 3DMigoto Deferring Texture2D dump: D:\Steam\steamapps\common\ONE PIECE BURNING BLOOD\FrameAnalysis-2023-10-02-230649\ctx-0x00000189E338C790\000045-ps-t2=4063c42d-vs=70a701dd177cd067-ps=4f6da09bd0afde2a.XXX
000045 3DMigoto Deferring Texture2D dump: D:\Steam\steamapps\common\ONE PIECE BURNING BLOOD\FrameAnalysis-2023-10-02-230649\ctx-0x00000189E338C790\000045-ps-t3=3396d2da-vs=70a701dd177cd067-ps=4f6da09bd0afde2a.XXX
000045 3DMigoto Deferring Texture2D dump: D:\Steam\steamapps\common\ONE PIECE BURNING BLOOD\FrameAnalysis-2023-10-02-230649\ctx-0x00000189E338C790\000045-ps-t5=a7818b71-vs=70a701dd177cd067-ps=4f6da09bd0afde2a.XXX
000045 3DMigoto Deferring Texture2D dump: D:\Steam\steamapps\common\ONE PIECE BURNING BLOOD\FrameAnalysis-2023-10-02-230649\ctx-0x00000189E338C790\000045-ps-t7=dbffa374-vs=70a701dd177cd067-ps=4f6da09bd0afde2a.XXX
000045 3DMigoto Deferring Texture2D dump: D:\Steam\steamapps\common\ONE PIECE BURNING BLOOD\FrameAnalysis-2023-10-02-230649\ctx-0x00000189E338C790\000045-ps-t8=dbffa374-vs=70a701dd177cd067-ps=4f6da09bd0afde2a.XXX
000045 3DMigoto Deferring Texture2D dump: D:\Steam\steamapps\common\ONE PIECE BURNING BLOOD\FrameAnalysis-2023-10-02-230649\ctx-0x00000189E338C790\000045-ps-t14=57fd2175-vs=70a701dd177cd067-ps=4f6da09bd0afde2a.XXX
000045 3DMigoto Skipped 3DMigoto resource in slot ps-t120

[HazzyDevil]

That could be down to a bad frame analysis dump - looking at the log file for draw call 45 I can see that it replaced vb1 and ib before the dump, and vb1 in particular wasn't backed up and restored (vb0 was). This is also evident in that the vb1 filename 000045-vb1-vs=70a70....txt lacks a hash after vb1, indicating that it was a buffer created by 3DMigoto rather than the game. Something might still be amiss here, but we should make sure we are looking at a clean dump with no replacements first.

It's generally a good idea to add a test on !frame_analysis in the ini files to prevent switching meshes when frame analysis is active to make sure the dumps will be clean. In DOAXVV I use this test (you can probably drop the $dump_modded_meshes part, that's there for situations where I specifically do want to see the results of replacing meshes in the dump):

if $costume_mods && (!frame_analysis || $dump_modded_meshes)

The log file also shows this matched both [ShaderOverrideLuffyClothes-Hat2-VS] and [ShaderOverrideLuffyClothes2-PS] causing it to run through [commandlistgeneral] twice, which might not be what you want.

Okay problem solved. I realised my own script which is meant to correct the semantics was also changing any mention of pointlist to trianglelist which ended up importing the drawn mesh as a mess of triangles as opposed to the individual vertices. I've also added that !frame_analysis condition just to be safe.

Now on the topic of semantics and the reason why I was using my script (excluding the pointlist to trianglelist function which was just me testing things and I have since removed it), I noticed that the names of each semantic are not in all uppercase. So instead of it being "POSITION", it's being dumped as "Position". "NORMAL" is coming up as "Normal" etc.

Which also reminds me, for another game, I've noticed that the weights dump out as "WEIGHT_VALUE" and "WEIGHT_INDEX", instead of "BLENDWEIGHTS" and "BLENDINDICES".

As a result, the blender script doesn't recognise these semantic even though they are just being named differently. Whether it's the same name but not in all uppercase, or if it's just a different word like weight_value for blendweights like above. So maybe worth adding these conditions in the blenderscript just so it's not explicitly looking for all uppercase semantics?

[DarkStarSword]

The official semantics that DirectX knows about are listed here: https://learn.microsoft.com/en-us/windows/win32/direct3dhlsl/dx-graphics-hlsl-semantics

However, as it states "In general, data passed between pipeline stages is completely generic and is not uniquely interpreted by the system; arbitrary semantics are allowed which have no special meaning"

Outside of the official semantic names, the script really can't make assumptions about what custom semantics are what, and stores anything it doesn't recognise in a vertex layer... however, to some extent it also can't really make assumptions even about the official semantics, because sometimes games just lie. I think it's ok to alter the semantic names in the frame analysis dump or even make a game specific version of the script to compensate for this for now, but I'm not convinced that it's a good idea to start adding lists of possible alternatives names in the generic script (making it case insensitive may make sense).

I could potentially add some sort of semantic remap feature to deal with these cases on import, if I can get the Blender UI to play nice (I don't want to pop up a dialog asking how to remap semantics for every file being imported, because that would make it tedious to import more than 2 or 3 files in batch. I might be able to add some extra fields to the import dialog, but then I can't do sanity checks that the types make sense and it's easy to forget to fill them out / make a typo).

[HazzyDevil]

Damn that sucks, but fair point. If there isn't consistency and game's can even 'lie' about what a semantic is being used for, then probably best to leave it up to modder. All good then, I'll just continue using my script and eventually integrate it with the blender script.

[DarkStarSword]

As a result, the blender script doesn't recognise these semantic even though they are just being named differently. Whether it's the same name but not in all uppercase, or if it's just a different word like weight_value for blendweights like above. So maybe worth adding these conditions in the blenderscript just so it's not explicitly looking for all uppercase semantics?

I've updated the script to treat these case insensitive for another game someone else is looking at - it won't solve the cases where they use a different name, but it will hopefully help for games that just don't capitalise them properly. If you give this a try let me know how it works out for you or if you run into any problems (seems ok in my testing, but I haven't thoroughly tested all the possible edge cases).

[HazzyDevil]

I've updated the script to treat these case insensitive for another game someone else is looking at - it won't solve the cases where they use a different name, but it will hopefully help for games that just don't capitalise them properly. If you give this a try let me know how it works out for you or if you run into any problems (seems ok in my testing, but I haven't thoroughly tested all the possible edge cases).

Tried the script and yeah it looks to import the meshes with lowercase semantics. Haven't noticed anything yet, but I'll let you know if I stumble across something.

[DarkStarSword]

Which also reminds me, for another game, I've noticed that the weights dump out as "WEIGHT_VALUE" and "WEIGHT_INDEX", instead of "BLENDWEIGHTS" and "BLENDINDICES".

Try the update I've just pushed up, which adds support for remapping semantics in the import dialog:

This is still very bleeding edge, so please let me know if you run into any problems / regressions with it.

[HazzyDevil]

Try the update I've just pushed up, which adds support for remapping semantics in the import dialog

This is still very bleeding edge, so please let me know if you run into any problems / regressions with it.

Okay so I installed the script, but I'm getting the following error when trying to enable the script on Blender v2.93.18

Traceback (most recent call last):
  File "C:\Program Files\Blender Foundation\Blender 2.93\2.93\scripts\modules\addon_utils.py", line 351, in enable
    mod = __import__(module_name)
  File "C:\Users\Hazzy\AppData\Roaming\Blender Foundation\Blender\2.93\scripts\addons\blender_3dmigoto.py", line 41, in <module>
    from bl_ui.generic_ui_list import draw_ui_list
ModuleNotFoundError: No module named 'bl_ui.generic_ui_list'

[DarkStarSword]

Hmm, looks like that helper was added around about version 3.5. I can add a backwards compatibility check, but I think I'll just disable the feature on old versions of Blender rather than bringing in the extra code necessary to add a list widget without that.

[HazzyDevil]

Okay decided to get Blender 3.6.5 LTS and yeah that resolves that issue. Now I'm testing the script on Unlimited World Red and the remapping feature works great. But I've noticed something, WEIGHT_INDEX (BLEND_INDICES) is in the format of R32_UINT instead of the expected R8G8B8A8 for this particular game. So upon importing into Blender, a mesh will have thousands of vertex groups, majority of which hold no value in weights. Seems like an easy fix though as my script previously would just do a simple conversion on the format as per below:

# Converting R32_UINT to R8G8B8A8_UINT
    infile = open(file,"r")
    lines = infile.readlines()
    i = 0
    for line in lines:
        if line.find("WEIGHT_INDEX") != -1 and line.find("Semantic") == -1:
            # Storing R32_UINT value for conversion
            x = line.split()
            y = int(x[-1])

            # Converting to 4 bytes (R8G8B8A8_UINT)
            y = y.to_bytes(4,"little")
            a = str(list(y))
            a = a.strip("[]")

            # Removing the R32_UINT value from line
            z = line.rsplit(" ", 1)[0]

            # Appending 4 bytes value to line
            line = (z + " " + str(a) + "\n")
            lines[i] = line

        else:
            lines[i] = line

        i += 1
    infile.close()

Reference: Unlimited World Red.zip

[AkaShrug]

Hmm, looks like that helper was added around about version 3.5. I can add a backwards compatibility check, but I think I'll just disable the feature on old versions of Blender rather than bringing in the extra code necessary to add a list widget without that.

while on topic of updating addon i wonder if you had any issue when trying to import full framedump folder with load related files checked , most of the time files count end up beging huge number like 100k sometimes which result in addon not even loading anything since it is still trying to match files , wonder if changing to set instead of list could help here or maybe this kind of use was not intended for it ?

https://github.com/DarkStarSword/3d-fixes/blob/97ee9073378e5ef9a23e71b3a0bbd77209031205/blender_3dmigoto.py#L1820

[DarkStarSword]

while on topic of updating addon i wonder if you had any issue when trying to import full framedump folder with load related files checked , most of the time files count end up beging huge number like 100k sometimes which result in addon not even loading anything since it is still trying to match files , wonder if changing to set instead of list could help here or maybe this kind of use was not intended for it ?

Thanks, I've implemented this suggestion as well as an additional check to avoid redundantly searching for files of the same draw call if multiple files of that draw call (i.e. index buffer + vertex buffer(s)) were selected in the import dialog

But I've noticed something, WEIGHT_INDEX (BLEND_INDICES) is in the format of R32_UINT instead of the expected R8G8B8A8 for this particular game

This can alternatively be achieved by changing the format type in the vb headers and importing with "load .buf files instead" enabled (the new "limit to draw range" option is also recommended in this case to match the behaviour of loading .txt files). I can also look into adding an explicit format reinterpret feature to handle cases like this that should be able to cope with loading the .txt files as well.

[AkaShrug]

Thanks, I've implemented this suggestion as well as an additional check to avoid redundantly searching for files of the same draw call if multiple files of that draw call (i.e. index buffer + vertex buffer(s)) were selected in the import dialog

seems better now , i was thinking maybe there is better way than using glob inside loop but im not sure if doing it differently is even better , like building list of all files in that directory and filtering from that but i have not done much testing to confirm and my machine randomly have issues related to disk so it is not reliable to test

[HazzyDevil]

Hey @DarkStarSword, wanting to get your thoughts on this.

I've attached a zip that includes Nami's body for your reference which includes both the posed mesh (that lacks weights), and the t-posed model that has the weights but is in pointlist topology.

What would be the best way to handle this so that the t-posed mesh with weights, doesn't use the pointlist topology and instead uses information from the posed mesh?

Nami.zip

[leotorrez]

Hey @DarkStarSword, wanting to get your thoughts on this.

I've attached a zip that includes Nami's body for your reference which includes both the posed mesh (that lacks weights), and the t-posed model that has the weights but is in pointlist topology.

What would be the best way to handle this so that the t-posed mesh with weights, doesn't use the pointlist topology and instead uses information from the posed mesh?

Nami.zip

We handle this in a rather hardcoded way in genshin. We simply combine the information of both buffers ourselves and then split the information again when we want to export it. The correlation between the two meshes is the vertex order, even if posed/unposed they correspond.

The blender plugin has an experimental feature to deal with this, is rather slow and very much in testing. But you can enable it by uncommeting the self.layout.prop(operator, "load_related_so_vb") line in the export class. For genshin's case we made our own version of the script as well but might not fit the needs for your game, in case you are interested: https://github.com/leotorrez/XXMITools/releases

Keep in mind this is a heavily modified version intended to be used with our custom pipeline. It is also a unity game so the tools might happen to work for your use case. Feel free to contact me if you need a hand testing it