talkinghead new features

[Technologicat]

Here's the first batch of new features for talkinghead:

• Rewrite sway animation (much more natural now).
  • Use all non-morph axes: head, neck, body.
  • New history-free, rate-based formulation.
  • Every few seconds, randomize a new deviation from the target emotion pose for all sway axes.
  • Assign this as the target pose, and let the pose interpolator perform the actual animation.
  • This gives us a non-linear, saturating exponential animation trajectory for free.
  • Micro-sway: add small dynamic noise (re-generated every frame) on top of sway target pose.
  • This makes the motion look more natural, especially once we are near the target pose.
• Add breathing animation.
  • How noticeable this is, depends on the character.
• When entering "confusion" emotion, allow blinking quickly in succession (for a short time).
• Add framerate limiter, currently hardcoded to send ~25 FPS.
  • In practice, the output FPS is a bit under this, because we don't account for the time it takes to pack and send the frame. This could be improved in the future.
  • Reduces average GPU usage if the GPU renders faster than this.
• Separate the render and network code.
  • Only generate as many frames as are actually consumed.
  • Add separate FPS counters for render and network send.

Also, some more code cleanup.

On an RTX 3070 Ti mobile, available render performance (with separable_half) is now ~50 FPS at factory clock rate (1700 MHz), and ~40 FPS when underclocked to 1100 MHz. Actual network send performance, after the rate-limiting to at most 25 FPS, is around 21 FPS.

Remaining TODOs are marked into talkinghead/tha3/app/app.py, there's a TODO list at the beginning.

I'll have to look at improving the network send next (I'd like it to produce a smooth 25 FPS, since there is definitely enough render performance for that), but I thought I'd post the new features now.

There are also some client-side bugs and missing features I might look into.

@Cohee1207: Comments?

EDIT: The plugin is now ~600 lines, up from the ~400 of the previous PR, due to the improved animations and the framerate limiter. I squash-merged all my talkinghead-next commits into this PR and rebased it onto the latest upstream/main, so it's clean as well as easy to merge. All important changes are summarized above. The extra two commits below fix minor omissions. From there on, anything new will go into a new branch to avoid messing this one up. :)

[Technologicat]

No comment this time? :)

Thanks for the merge, and merry xmas!