model position in world space (?)

[AkaShrug]

can addon import models in same positions as world space (im not sure thats what it is called ) i only know about pose cb import but not sure if thats what needed here or does 3DMigoto Framedump have such data ? i only checked few places but dont think i found anything my end goal is just being able to import framedump with all models positoned (location) in same way as in-game

[DarkStarSword]

Generally not - frame analysis can only dump the buffers that are passed to the GPU, while the conversion to world space occurs in the vertex shader.

There are two ways this could conceivably work:

1. Examine the vertex shader and look for where it's local to world matrix (or whatever the game named it) is located, which will usually be somewhere in one of the constant buffers - but note that it may not exist at all, as many games skip over world space entirely and go straight to view or clip space. In some cases if the matrix is missing you may still be able to derive it from other matrices with a bit of maths (e.g. MVP * inverse(VP) = M).

If you are able to find a suitable matrix, dump out the constant buffer it is in with frame analysis, then modify the blender addon to multiply the coordinates by that matrix (which might be in row-major or column-major order changing which way around the multiplication goes).

There are probably a few too many game specific aspects of this to realistically add this to the generic script, but maybe we could add an optional field for it in the importer that could work in some games.

2. The other way it could conceivably work would be to extend the frame analysis feature to use the stream output facility in order to dump the geometry after it has gone through the GPU. This isn't supported at the moment, and even if I were to implement it, you would wind up with the geometry in clip space (so, heavily distorted) which would probably not be very useful, though it could theoretically be converted back to view or world space by multiplying it by the inverse [view]-projection matrix... which we would need to find in the same manner as option 1.

[SilentNightSound]

Hi xodd! I figured I would post my response to the question here so the response would be visible.

For unity games that do posing in the GPU and have the Skinning/Draw split into different calls (e.g. all Mihoyo games), there are specific VS that draw the model to the screen and contains information on the world location of the model - I believe in Genshin, that data is stored in CB2[0]-CB2[5] and CB3[21] (one is relative to the world, while the other is relative to the current camera location, can't remember which is which at the moment) and there are ~5 or so VS that contain the information (the ones that draw the model).

Using that in conjunction with the pose data it is possible load the "true" position of the model in blender - I actually did something similar when getting the animations to work, since replacing the pose data alone isn't sufficient (since even with the new animations the model will still rotate/move as the character breaths/attacks). This could be extended to actually load custom animations into Blender from the frame analysis folders, though I only got the first portion of that (pose) working and never managed to implement the world location portion.

Generalizing this to other games would be tricky since they may have different methods of calculating where the model is located (they might not use the same formula, though I'm guessing the form would be fairly similar since it is just a matrix multiplication).