Load Mixamo Animation Animation

[WaltzBinaire]

Hey im rigging and animating my character with mixamo...then i import them to blender via FBX ...i then export them to js with your exporter (tried every exporter ...every).

The result is here: http://instagram.com/p/y8NQjQkk9-/?modal=true

it always looks distorted like this and i cant seem to understand it...its always the hands...Is it because the number of bones. mixamo has 16 or sth. and the marine_anim from the examples more then 50.

Please help a brother out...after three days im at the end of my wisdom|tinkering

[repsac]

After this PR https://github.com/mrdoob/three.js/pull/6052 is merged I suggest trying the exporter from the dev branch. It has 2 different pose position options for POSE and REST. Set your armature to one or the other, and then during the export make sure the Skeletal animation property matches. Try both and see if either helps. For some people POSE works, and others it is REST.

[WaltzBinaire]

hmm so exported using skeletal animation rest and in pose mode...

this are my files http://we.tl/CNM5qEmBMs including orginal finale and both exports...

Rest mode = no movement at all Pose mode = weird distortions

u said " It has 2 different pose position options for POSE and REST.Set your armature to one or the other " ...i dont know how to set my amature to pose or rest...i want to use blender only as a translator from mixamo to threejs...sorry if my question is to noob

[WaltzBinaire]

did you mean the editing modes pose and edit mode for objects?

[WaltzBinaire]

Ok Pose mode ---same distortions Edit mode the following error log:

[repsac]

Can't recreate this error but I noticed, by looking at the line numbers in the traceback, you're possibly not on the same (latest) version found in dev. Also make sure you're not doing a Scene export with this asset. Just do single geometry exports for now (since ObjectLoader doesn't work with skinned meshes yet).

As far as broken animations are concerned; I can recreate both the broken animation and the missing animation issue. There are people getting working animations exporter (example https://github.com/mrdoob/three.js/issues/5582#issuecomment-73444737) so I am suspicious that there are some issues with FBX import process. It is very hard to determine if it is the exporter or the FBX import. I wondered if there are transformations that needed to be applied to the bones, but I don't yet know.

[WaltzBinaire]

Hey man thank you a lot for your time and insight!

you're possibly not on the same (latest) version found in dev.

I think im working with the latest version because i can select pose and rest: "Animation option is now an enum that requires a user to select between POSE and REST"

since ObjectLoader doesn't work with skinned meshes yet

From my experience it doesnt madder...i loaded geometries and scene with objectloader into threejs. The following error is created with Objectloader and gives same result as previous Geometrie exports.

I can recreate both the broken animation and the missing animation issue.

It would be really interesting if you are NOT able to recreate the issue... Hence do you see any possibilty to export the model to threejs...

Its interesting in general as its a model exportet by mixamo. If this model works - every mixamo model would work. (in theory ;))

anyway thx for looking into it.

[WaltzBinaire]

With the current version of Blender and only geometries and no scene export i get in pose mode:

and in rest mode

i can only export the geometry if i export the hole scene

[WaltzBinaire]

ok i think i found it...its a bug in the reference between vertices of the geometry and the bones animated...mesh and skelet is off

After manually scaling and rotating the mesh towards the skeleton like this

for (var i = 0; i < geometry.children[0].geometry.vertices.length; i++) {
                vector = geometry.children[0].geometry.vertices[i];
                var axis = new THREE.Vector3(0, 1, 0);
                var angle = Math.PI / 2;
                geometry.children[0].geometry.vertices[i].applyAxisAngle(axis, angle);
                geometry.children[0].geometry.vertices[i].multiplyScalar(0.01);
 }

i got a quite ok result.

So first of all - why do i need to rotate the mesh...i remember when importing to mixamo that i had to rotate my model exactly 90 degree...its because of -z directing Why scaling? no clue...

now comes the buggy thing my model didnt had the perfect TPos...the arms were to low...i rigged it in mixamo and it was still perfect afterwards...i saw it in blender ...and it was perfect to...saw it in max was perfect too. THEN the export to Threejs and the mapping between bones and body was reffering to the originally created body in max.

hence there is a math bug in referencing vertices and bones in their transformation matrix.

[repsac]

I wonder if it is actually a bug or instead an optional feature. This could very well fix the issue you're having but break exports for someone else who is not having to import FBX assets (something I have seen before already with POSE and REST animations). The logic being used (for bones and vertices) has been in place for awhile (copied from the old exporter) and I am not saying that it is perfect or without flaws but if this was a bug in the math I think we would have seen more issues. Have to apply your code example into the exporter and then run lots of tests on lots of test cases (which isn't something I have lying around).

I have seen similar problems in production where an asset is making multiple trips through multiple programs and exporters. Add into this equation axis re-orientations and you have a recipe for certifiable shenanigans. If I can figure out how to get this implemented into the exporter it will be an option only until there is far more certainty that it does indeed apply to all users and not corner cases.

[repsac]

With the current version of Blender and only geometries and no scene export i get in pose mode:

The error specifically says "Not a valid mesh object". Geometry exports are selection based and you most likely have the armature selected and not the mesh.

[WaltzBinaire]

hey man again thank you for looking into it. i didnt wanted to say that my code should get into exporter. What i wanted to point out is : that within the chain from max to mixamo to blender - everything gets assigned the way it should be. once it gets to threeJs - there is a simple transformation offset happening (in my case it was rot and scale) which cause the error.. this transformation is reffering to the original mesh that was once created (where ever) and is ignored in blender and mixamo...but revived during the export.

unfortunatly i have to admit that i am not a matrix magician - otherwise i would love to give a helping hand as you guys are working super hard. Thank you for your insights.

[repsac]

I never used mixamo. Does it use Y-up? What I wonder what happened is that if maximo is Y-up then the FBX import would have to reorient the asset to Z-up, and then the exporter reorients again for Y-up to three. Wonder if the asset's transformations would need to be applied in Blender upon import if that would then export correctly to three.

Translating rigs from one platform/format to another gets ugly after more than a couple iterations. Even in some of the most advanced pipelines and tools I have ever worked with there was always limits on what a rigged asset could be put through. There may be a solution out there that works for you and everyone but it certainly isn't immediately clear what it is.

[repsac]

i didnt wanted to say that my code should get into exporter.

And I was more generally referring to something that would attempt to apply axis angles to vertex positions.

vector = geometry.children[0].geometry.vertices[i];
var axis = new THREE.Vector3(0, 1, 0);
var angle = Math.PI / 2;
geometry.children[0].geometry.vertices[i].applyAxisAngle(axis, angle); <-- would need something like this as well

but I am also grasping at bit, and maybe this is trying to make the exporter handle too many variables

[repsac]

Interesting comment here concerning FBX https://github.com/mrdoob/three.js/issues/2803#issuecomment-11458920

[please-wait]

Have you applied the rotation you've probably made to your model and the armature inside the blender file? You know, with ctrl + A > Rotation and Scale, in object mode. I've seen very similar stuff happening because of not doing that. @princemio

[kimasendorf]

My mesh comes constantly out distorted or I get errors and I am not able to export multiple animations. Switched back to the r69 exporter that seems to work well even in Blender 2.73a

[WaltzBinaire]

@please-wait ...no actually i didnt...but maybe from mixamo to blender... @kimasendorf did you tried to manualy recreate the mesh in threejs?

[kimasendorf]

The mesh was modeled in Blender. Even though it is a low poly character it is certainly to complex to manually recreate the mesh in threejs. And how could that help?

[WaltzBinaire]

in my case mesh and bones where not aligned with each other...hence the model bend trying to fit the bones...if you dont have bones then this hasnt anything to do with it...

[WaltzBinaire]

btw...i dont mean "recreate" lol...im sorry i tried to write "realign" the model...

[antont]

I got similar problems with every more complex skeleton I tried so far. Including the marine included in three.js repo and featured in the examples. Here is what I get with that using the pose switch:

Marine in pose: https://antont.github.io/three.js/examples/webgl_animation_skinning.html

That export is in: https://github.com/antont/three.js/blob/blenderexport_buggy_marineskinning/examples/models/skinned/marine/marine_anims-pose.json

The minimal example from exporter tests works. The skeleton helper matches the model when using pose mode there.

I'll try (some of) these tricks from that already closed other issue (#4660) next:

1. You will need keyframes on every bone you want to export.
2. The mesh needs to be in the T-pose. Here are two ways to do this: a. Have a T-pose action. This action needs a frame where all bones have transforms cleared. This should be the current frame when the exporter is selected. b. Delete the Armature modifier from the mesh immediately before export. After the file is exported, undo the last action to get the modifier back.

and:

I could only get this working when I kept the Armature modifier, but disabled both vertex groups and bone envelops (then re-enabled after export). In my case, deleting the modifier didn't help.

perhaps also:

Have you applied the rotation you've probably made to your model and the armature inside the blender file? You know, with ctrl + A > Rotation and Scale, in object mode.

If someone beats me in getting a working export from that example blend and tells what was needed .. many thanks! :) Am using this blend: https://github.com/mrdoob/three.js/blob/master/examples/models/skinned/marine/marine_anims.blend

[antont]

Ok got it working now, updated https://antont.github.io/three.js/examples/webgl_animation_skinning.html (if someone saw the buggy version earlier be sure to not get it from cache now)

Tried a lot of things back & forth, checked all bones were keyed, both the rest & pose options etc. but in the end it seems it was just that when I had apply modifiers on got the weird deformation. Makes sense because then the armature modifier was applied to the base geom at export, and then further deformed by the skel anim in three.js.

So there's a working export of the unmodified marine blend with the new exporter at: https://github.com/antont/three.js/blob/example_skinning/examples/models/skinned/marine/marine_anims.json

I think it has only one of the animations though, need to figure out next how to export all anims with the new export.

Screenshot of the options used:

[mrdoob]

@antont thanks for researching this!

[antont]

the deer from realxtend / chesapeake bay works too :)

same options.

demo: https://antont.github.io/three.js/examples/webgl_animation_skinning-deer.html export: https://github.com/antont/three.js/tree/example_skinning_deer/examples/models/skinned/deer source: https://github.com/realXtend/chesapeakebay/blob/master/objects/White_Tailed_Deer/White_Tail_Deer_all.blend

[repsac]

I setup a Slack team channel for three.js, mainly to work on animation issues but it can be used for anything three related. This is a private team channel so people can only join by invite. I am looking for:

• people that know Python
• people with a solid understanding of rigging in Blender
• people with animation experiences and scenes willing to share

This is not an elitist thing. I am limited on time and don't have bandwidth to teach someone Python or Blender, so I apologize if this is does not seem newbie friendly (that's what IRC is for). This is for those who actually want to nip this in the butt and have some useful skillsets to contribute. IRC annoys me and even github threads aren't my favorite place for collaboration. Been using Slack with other groups and it has been great.

Send me an email (and please note your github handle too) for an invite ed[dot]caspersen[at]gmail[dot]com

[manthrax]

I think a change in the latest blender apis have broken the method that the exporter uses to select the action on the mesh.

The behavior I'm seeing (not completely confirmed) is that the exporter is exporting:

1. The action that is selected on the armature that is the parent of the selected mesh...
2. Attempting to export the other non-selected animations, and failing when attempting to set the animation on the armature, in order to step through the actions keyframes. The result is the the animation from step 1 is exported repeatedly under the different action names... The duration does seem to be correct however amongst the different actions.

I think a recent change in Blender now requires the armature to be the actual selected object in the 3d view in order for action changing to actually take effect.

This could be solved by changing the exporter to require the armature to be selected for export, instead of the mesh, and the action changing performed int the script may work.

Let me know if you want more help with the blender exporter.. I have a decent background in the scripting for pre 2.6..

[manthrax]

Another issue, I had to have the right combination of "rest" selected both in the script params and on the armature. This seems brittle..

Another problem is the "Apply Modifiers"... From what I can tell, in the normal circumstance of exporting a skinned mesh, this should Never be the case, and should default to off or have a separate toggle to exclude skinned meshes. Applying modifiers, makes the armature transform the mesh to whatever frame it currently is set to. This hoses the binding for the mesh and demonstrates the problems shown elsewhere in this thread.

Would be nice to find a way to streamline these steps.

[manthrax]

I just ran some tests and confirmed that the exporter is exporting the frames of the same animation for each animation exported...

So since I exported my mesh with the "walk" action selected... All the animations in the file are copies of the "walk" animation under different names and with different lengths.

I'm surprised the exporter isn't outright crashing in the case that you have an action with more frames in it, than the other animations in the file.

[manthrax]

A succint description of this bug:

All exported animations keyframes are those of the selected animation instead of their own keyframes.

[manthrax]

I have found at least a partial fix for this...

In "io_three\exporter\api\animation.py"

at the beginning of "parse_pose_action" (around line 183)

insert:

context.scene.objects.active = armature

With this change, I now get unique animations exported...

[derivou]

Hi, thanks for this feedback + fix. I made the recommanded change and exported with "Pose" and "Apply modifier"=off and this works great !! Thanks

[titansoftime]

context.scene.objects.active = armature

Looking forward to trying this.

[MattTarquin]

Had the same issues which I raised in:

https://github.com/mrdoob/three.js/issues/7165

Work around worked for me, happy that I can actually get multiple animations going. Thanks for that!

[manthrax]

Great! Glad I could help!

[manthrax]

I linked some other bugs that may be manifestations of this, back to this thread.

[arturitu]

@manthrax (with your partial fix) it works well for me exporting 'Skeletal animations: Pose' and deselecting 'Use modifier during render' in the armature modifier.

A demo of a character with 3 animations working > http://unboring.net/lab/6326/hero_72/

Do you think it would be possible/easy to export and run animations correctly with 'Rest mode'? Because in that case, the exporter only write the keyframes not every frame, and the json file goes from having 873kb to 174kb

[repsac]

Sent out some Slack invites. Not sure what kind of time I will have tomorrow but should have a couple hours on Sunday.

[manthrax]

@arturitu I can look into that... What is the behavior right now for rest_mode pose that is incorrect? Does it seem like the same issue?

@repsac:

Exporting currently takes a long time on my machine for large animations...

It turns out that the vast majority of the time is due to log spam...

And simply turning off the logger probably wont fix this. I have had this problem with python scripts in the past... Even if the logging output is disabled, the string construction still takes place in the logging statements. The script will need to be re-written to do something like "if(logging)log.debug("blah");" on all logging statements, in order to not incur the overhead of formatting all the messages.

Simply commenting out the logging messages I saw in animation/keyframe export, made my export time go from 2 minutes, down to around 5 seconds.

[manthrax]

@arturitu I tried the rest_pose export... I did get a smaller file, but the skeleton bindings are screwed up when attempting playback... Any ideas?

[manthrax]

I'm testing with the marine blend file, bound to a different mesh...

Skeletal animations: Rest

[manthrax]

Skeletal animations: Pose

[arturitu]

@manthrax I think in Rest has a problem with the bones part on JSON, because a lot of them have a lot of "parent": null, while in Pose the bones have the correcto parent value. And the number of bones is different ( in my case in Pose write 38 and in Rest only 28 )

But I don't know if it would be necessary other kind of BlendCharacter.js to show a correct animation

[repsac]

The script will need to be re-written to do something like "if(logging)log.debug("blah");" on all logging statements, in order to not incur the overhead of formatting all the messages.

Probably not, this may be addressable within the logging module itself. Look at that now

Another issue, I had to have the right combination of "rest" selected both in the script params and on the armature. This seems brittle..

It is, and I had attempted to solve this in the past. There are commented out sections with

armature.data.pose_position =

all over. When trying it in Blender's shell it worked but it wouldn't during export. I haven't tried in recent Blender versions so maybe this was fixed (until it is broken again).

With this change, I now get unique animations exported...

Good catch. Blender's contextual awareness that it requires is so implicit at times it becomes just maddening to deal with.

[repsac]

Log disabling was easy. Since three's repo is just too bloody large I created a temp repo that will only pertain to the exporter, everything will eventually get merged back in. https://github.com/repsac/io_three

From what I can tell, in the normal circumstance of exporting a skinned mesh, this should Never be the case, and should default to off or have a separate toggle to exclude skinned meshes.

If a good case can't be made why it should stay on then I would elect to set it to False if animations are enabled. The only issue may be in scene exports, which I am on the fence if scene exports should support animations anyways. Have had issues with how Blender handles actions under the hood that makes it seem very fragile.

[manthrax]

@repsac Just wanted to say.. The exporter is awesome. I hope none of my comments come off as criticism... I have written much more primitive blender exporters in the past.. but nothing with the scope of this one, so it's no stretch to say it's a big achievement.

r.e. Logging: I am still not sure how the overhead of log message formatting is eliminated by just disabling the output in the logging framework, but maybe I don't understand the nuances of the framework.

From what I can tell, any logging of the form "someLoggingFunction( "formatted text:x y z")" will ALWAYS incur the text conversion/formatting of the message, even if the function does not emit the text internally

For zero overhead logging, things would have to take the form of "Logger.log( function(){ return SomeFormattedText; }), where the passed in function is only called if logging is enabled.

. In a truly compiled and optimized language, the compiler might be able to detect that the function is a no-op and sklp the formatting calls, but even then, the formatting calls could have side effects for the rest of the program, so generally they would still be included.

r.e: Scene exports supporting animation... I agree, that in the near term, it's probably too early to go down that road, BUT, if we get to the point where that could work.. it would be pretty amazing to be able to build entire levels with animation, and export WYSIWYG to three.js Windmills would spin... rivers and waterfalls could UV animate.. platforms could fly around.. all without a writing any code. Some day... :D

[repsac]

From what I can tell, any logging of the form "someLoggingFunction( "formatted text:x y z")" will ALWAYS incur the text conversion/formatting of the message, even if the function does not emit the text internally

Well any logging call should be formatted as such:

logger.info("foo bar %s", arg)

So the string isn't formatted until the logging API gets passed *args under the hood. I changed the logging logic to

https://github.com/repsac/io_three/blob/master/addons/io_three/exporter/__init__.py

        level = options.get(constants.LOGGING, constants.DISABLED)
        version = options.get('addon_version')
        if level != constants.DISABLED:
            logger.init('io_three.export.log', level=level)

https://github.com/repsac/io_three/blob/master/addons/io_three/logger.py

def _logger(func):

    def inner(*args):
        if LOGGER is not None:
            func(*args)

    return inner

@_logger
def info(*args):
    LOGGER.info(*args)

@_logger
def debug(*args):
    LOGGER.debug(*args)

@_logger
def warning(*args):
    LOGGER.warning(*args)

@_logger
def error(*args):
    LOGGER.error(*args)

@_logger
def critical(*args):
    LOGGER.critical(*args)

No message should be formatted. I don't have any large scenes so I can't test this but in theory it should, and if it doesn't then that becomes reason #4576 why I hate programming.

I hope none of my comments come off as criticism

Not hardly, between tech companies full of bro-grammers here in SF (including Silicon Valley) and the VFX industry I have had waaaaaaay worse said to me in interviews.

Currently going over this whole thread and trying to put together a bullet list of what specifically should be done. Will post it when I have something so you can see if it is close.

[manthrax]

Oh excellent! Looks like you've got the logging covered. My python was a little weak to understand the nuances of your solution.

Small world, I'm in SF as well!

Very cool... as far as what I can glean from this thread, it's mostly the one line fix, and then resolving the ambiguity of the "rest" mode... I'm still not sure what usage case rest mode ever has.. if you know of one, I'd be grateful to understand it...

[repsac]

I would have to dig through all the old issues to give you specific links but for the sake of a quick response here is a break down:

• around r66 there was a commit that changed exports from using REST to using POSE mode
• I had begun my own version of the exporter around this time, which became the one we are talking about now, and based my original logic off the REST logic
• at r70 mine was the replacement exporter, that is when all hell broke loose
• I replaced the logic with POSE support, which then broke exports for other users
• I couldn't make sense what the hell was going on so I implemented the toggle between the 2

So far there seems to be 2 items to address:

1) setting the active context on the armature

context.scene.objects.active = armature

2) don't apply modifiers to skinned meshes

I will test the automated pose switching again. Someone once said they had that figured out from a different script but that secret sauce didn't make it into the exporter.

[repsac]

@manthrax Can you checkout my branch and see if this is working for you? https://github.com/repsac/three.js/tree/anim_fix

[MattTarquin]

This is working well for animations in pose mode as long as your first keyframe has the armature configured exactly the same as your mesh. If you try to add any loc/rot/scale transforms, in pose mode, and assign them as the first keyframe then the model renders distorted.