Skeleton's joint model space to world space position

[Paltoquet]

I am currently trying to replace an assimp loader with the ozz library. The end goal is to provide for each key frame a dual quaternion for each bone of an animation, the actual "skinning" is done on the gpu for rendering large crowds. I am using openGL with assimp and use column major matrix convention, vector = Matrix * vector. I define world space, or world position as the position of the vertex before applying the usual mvp matrix. I have several questions,

• Giving a sample::Mesh and its animation::Skeleton, how can I display the joint location in world space ?

I see how to retrieve the joint position in model space via LocalToModelJob but not what to do afterward.

// Stage 1
ozz::animation::LocalToModelJob ltm_job;
ltm_job.skeleton = &skeleton;
ltm_job.input = skeleton.joint_bind_poses();
ltm_job.output = make_span(models_);
ltm_job.run();

// Stage 2 Model -> World ? 
for (size_t i = 0; i < mesh.joint_remaps.size(); ++i) {
  skinning_matrices_[i] = model_space[mesh.joint_remaps[i]] * mesh.inverse_bind_poses[i];
}

• Can the world position of a vertex be expressed like this, or am I missing something ?

vec4 world_position =  models_[mesh.joint_remaps[i]] * mesh_.inverse_bind_poses[i] * vertex_position;

• How the difference between the sample::Mesh bind pose and the Skeleton rest pose is taken in to account in the ozz library ?

• Is there some utility function to convert math::Transform to math::Float4x4 ?

Thanks for your time

[guillaumeblanc]

Hi,

many questions there! Thanks for asking, it's probably interesting for other people.

I define world space, or world position as the position of the vertex before applying the usual mvp matrix.

I'm using a different wording, because to me the m of mvp is the matrix that transforms model space to world space. So I'd never compute vertex world space on the cpu, but let the vertex shader to the math (using mvp). Ozz sample does it this way for static meshes, and the same way for skinned meshes whose vertex model space is computed on the cpu.

Giving a sample::Mesh and its animation::Skeleton, how can I display the joint location in world space ?

Stage 2 is building the skinning matrices, aka the matrix required to transform a model space vertex by a joint transform. inverse_bind_poses purpose is to bring the vertex into joint space. You can find better explanations here for example.

You don't need the mesh to display joints in world space. You don't need bindpose matrix either. LocalToModelJob allows you to find model space location (or the full transformation actually). Joint location is the 4th column of every matrix filled by LocalToModelJob (models in your code). Multiplying model space joint transforms to the world matrix of your model, will give the word transform of the joint. Usually we don't compute world space location of a joint, because we send model space data to the gpu (like discussed above).

Now if you really need to compute the world transform of a joint, LocalToModelJob as an optional parameter that allows you to provide a root matrix. Model space matrices will be multiplied by this root matrix, so can provide your world matrix as the root to achieve your result.

Can the world position of a vertex be expressed like this, or am I missing something ?

Looks ok, if "i" is index of the joint transforms the vertex (which is not obvious in your example). Joint indices are available from the mesh interface.

You can refer to ozz skinning implementation, but it's not really readable because of macros.

Note also that this code works for a rigid skinning. If you have multiple joints influencing/transforming/skinning a vertex, then you have to blend them during skinning. As an example, this is the implementation for 4 influencing joints.

How the difference between the sample::Mesh bind pose and the Skeleton rest pose is taken in to account in the ozz library ?

Skeleton rest pose is a default joint transform. It has no real meaning, except that it can be used to display the skeleton when it's not animated, as a default pose.

Mesh bind pose is important for skinning. It refers the skeleton pose that was used when the mesh was skinned. It can be different from the rest pose. All the mesh skinning setup (joint weights and indices) is done for this bind pose. We use the inverse bind pose of a joint to transform a vertex into that joint's space.

Is there some utility function to convert math::Transform to math::Float4x4 ?

You can use Float4x4::FromAffine. There are examples in ozz code base.

By the way, there's a ozz implementation in Sokol. This can be a great source of example for you. This sample displays a skeleton, this one skins a mesh on the gpu.

Hope it helps, Guillaume

[guillaumeblanc]

Don't hesitate to reopen if something isn't clear.