Dual quaternion skinning

[elisee]

Hello!

Would Dual Quaternion Skinning (DQS) as described in this paper and showcased in this video be welcomed as a feature for Three.js? It seems to be reasonably simple while giving strong visual benefits.

From what I've gathered, in Blender, DQS is exposed as a "Preserve Volume" option on the Armature modifier.

From reading the aforementioned paper, it seems like support would require:

• Storing object-space bone rotation and translation (encoded as dual quaternions) into 2x4 GLSL matrix uniforms instead of using 4x4 bone matrices directly
• Some math in the skinning vertex shader chunk to interpolate those rotations and translations separately, normalize the dual quaternions and then recreate the transform matrices from that (or alternatively apply the rotation & translation directly on the vertex).

I assume we'd need some kind of switch between regular Linear Blend Skinning (LBS) and DQS. Not sure where it would be most appropriate to put it? As a property on SkinnedMesh?

I might give it a stab if someone experienced with Three.js's design has pointers on how to approach it.

[donmccurdy]

I think the difficulty here would be authoring models to use this feature... standard export formats don't support this type of skinning data, to my knowledge, and we're no longer maintaining a custom Blender exporter just for three.js. After loading a model that uses LBS, can you convert it to DQS safely at runtime, or does this have to be done when it's exported from Blender?

[elisee]

My understanding is that it's a fully-runtime change with no impact on data. Quoting from the paper:

In an existing application, it is extremely easy to replace a linear blend skinning implementation by a dual quaternion one. All that is necessary is a slight modification of the vertex shader and conversion of the matrices to dual quaternions before passing them to the shader

[mrdoob]

If the data stays the same and the results are better then a PR would be very much welcome! Would performance be affected though? (Haven't been able to check the paper/video)

[makc]

the results are better

shouldnt the results stay the same too? someone would spend time to animate the model in maya only to find out it is animated diffrently in 3js. this whole thing has to b optional

[lalalune]

+1 for this.

The reason for dual quaternions is that it is much more performant, especially for complex bone transform hierarchies... think about the upcoming rise in popularity of WebXR and avatar-centric content in general. DQ is always more performant, and usually more visually appealing for skinned meshes, I don't think there are any situations where you would't want this 100% of the time.

There are numerous things that make make for visual differences between exporter and runtime (# of weight influences, etc) -- any visual differences here would probably end up a net positive, especially for armatures with max 2 weights, and this would be a huge optimization for our application for sure :)

[mrdoob]

If performance is better and the result is better, sounds like something we should have. Anyone up for giving it a go?

[msub2]

I'm feeling adventurous; give me some time to study up on my math and work out the implementation details and I'll see what I can do.

[brainexcerpts]

Hi, I randomly stumbled upon this thread. I wrote a tutorial about DQS a while ago (http://rodolphe-vaillant.fr/?e=29) so if you need any help understanding the math I can give insights. @msub2 Keep in mind fully supporting scale transformations with DQS can be challenging contrary to LBS: (http://rodolphe-vaillant.fr/?e=78) And yes performance between LBS and DQS are mostly the same.

I'll have to disagree on the "switching over to DQS is always desirable". Talking from my actual real world experience my short answer would be: it depends... In some cases yes but mostly no.

The long answer would be: If a model was designed with LBS in mind chances are it won't look so good with DQS. This is especially true when the model was made by a skilled rigger with lot's of helper joints. LBS has problems with volume collapsing and artists/riggers know how to counter it with magic helper bones: those additional bones sometimes don't play so nice when you "just" switch over to DQS.

DQS on the other hand can have a tendency to bulge: (http://rodolphe-vaillant.fr/?e=72) which can be taken into account with proper skin weights. But if you designed a model with LBS in mind you could end up with over bulging of your skin as DQS reacts differently.

If DQS will allows for a much quicker and pain free design and weight painting of your rig, you need to build your rig with DQS in mind for optimal results

That being said simple models without helper joints and not too smushy skin will definitely benefit from a switch over to DQS.