Talkinghead next2

[Technologicat]

Here's an update to talkinghead.

• /emote support via new API endpoint /api/talkinghead/set_emotion (needs https://github.com/SillyTavern/SillyTavern/pull/1622 for frontend support)
• Additional performance improvements:
  • Run render/encode in parallel to get each new frame ready as soon as possible
  • Use PNG compression level 1 for much faster encoding (down to 20ms per frame, from 40ms at default 6); per-frame network send on localhost still under 0.15 ms
  • Hit the target output frame rate almost exactly
• Prepare for upcoming configurable pixel-space postprocessor (bloom/fake-HDR, scifi hologram, VHS tape, cheap video camera, ...)
• Cleanliness: add missing @require_module to talkinghead API endpoints

The postprocessing filters (essentially fragment shaders written in Torch) exist and work, but the postprocessor is temporarily disabled, because we're still missing a way to send it a configuration from the client side. If someone wants to experiment with it in its current state, look at talkinghead/tha3/app/postprocessor.py, and see default_chain, which is a simple data structure containing the (currently hardcoded) settings. Adapt the provided example as desired, and restart your SillyTavern-extras.

I think /emote xxx and this second round of performance improvements could be worth a merge already.

Live mode is still practically usable only on a GPU, due to the posing engine THA3 being a deep-learning model.

Comments?

[Cohee1207]

No comments. Looks fine as it is

[Technologicat]

Thanks for the merge!