LookAt, Gesture, and Emote samples?

[roschler]

I cloned this repo locally and then used a full text indexing package on its contents. I can't seem to find any samples that show how to use AWS scripting to do the following operations:

• Execute the Host component look-at feature (E.g.- look at camera)
• Execute the Host component gesture feature
• Execute the Host component emote feature

I need to be able to do these things from the new Preview Scripting environment, not the legacy scripting environment or from a IDE provided state machine. Where can I find sample code for these features?

[c-morten]

Hi @roschler. This repository makes Host functionality available to open source engines like three.js and Babylon.js, it is not compatible with the Sumerian engine as it already has hosts as a built-in feature. I would suggest reaching out to the AWS forums or Sumerian slack channel for help with the Sumerian scripting API: https://forums.aws.amazon.com/forum.jspa?forumID=300 http://slack.sumerian.aws/

[roschler]

Thanks will do. I thought this was a set of samples for AWS Sumerian native. Sorry. As far as the Slack goes, it's hit or miss over there. Some weeks responses come reliably. Other weeks, there are long periods with no answer. I've been trying to get answers on how to do the three features I mentioned above off and on for 2 months now and still no luck. The AWS "official" forums have, over the years, given me a response rate around 10% so it's just not worth the effort (i.e. - for every 10 posts I make I get one reply and not necessarily a helpful one).

[roschler]

@c-morten One other question. Is there any reason that I should stick with Sumerian native over what this repo offers? I have Three.js experience and I am not using Lex. Instead I do my own chat handling and I could easily switch to calling Polly directly because I already do that in other applications I have written. Also, I don't intend to publish a Sumerian scene, instead I will be beaming a session over Twitch.

Is this repo stable? Like I said, now that I understand what it does, thanks to your reply, I'm wondering if it's the better choice for me?

[c-morten]

The host functionality we offer here is at parity with what you'll find inside Sumerian, with the added benefit that you can use it with your own custom 3d content if you prefer. So the choice really depends on what other functionality you require. If all of the features you need outside of hosts have equivalents in three.js then the only benefit for using Sumerian would be the editor.

As far as stability goes, this is a new repository. We have tested quite a bit with our own assets but there's always the possibility for bugs once more people start using it since I'm sure it will be used in ways we hadn't originally considered. That being said, we monitor the issues and do our best to release fixes quickly when issues are reported. The npm package is versioned so it's always possible to get the a version you know to be working if a bug were to be introduced.

If you want to get up and running quickly to test this repository out, I would suggest cloning the repository and reading through the Getting Started in three.js steps in the readme. The steps walk you through an example implementation in three.js and should give you a general understanding of the API. We provide the finished example in the examples folder, all you need to do to see it working is add a cognito ID with permissions for Polly and run it on a local server.

[roschler]

@c-morten Excellent, I will do that.

What is the best sample for learning how to get the Host to look at another item realistically?

For example, suppose a Host was standing face forward and looking at the camera, and I wanted them to look at another Host standing directly along side of them (i.e. - to the left of them, also facing the camera). I don't want them turning completely sideways to face that other Host and thereby offer up only a profile view of themselves. Instead, I would like that Host to turn their head and then somewhat angle their body to look at the other Host, instead of doing a full body turn to face them. On the other extreme, I don't want them just facing the camera directly and just swiveling just their eyes to look at the other host. How would I do this with the usual tweening?

The overall context would be two Hosts facing the camera for a presentation, and one of them looking at the other in a natural manner when the other Host is speaking. Like a pair of newscasters.

[c-morten]

The example we provide includes a basic look interaction. There are 2 hosts and when one speaks, the other will look at the speaker. It uses the same type of point of interest functionality that we have in Sumerian, so you'll see the head/neck and eyes rotate at different speeds and to different extents, and the eyelid and brows move along with the eyes.

If you wanted to take this further, you could add more animations for turning the body and either add another layer to the point of interest configuration to drive those animations (you can look at the existing configuration for details on what inputs are required) or trigger body rotation animations manually based on conditions you define in your script. Although we focus on lipsync, gestures and point of interest in the example, you can trigger an animation at any time using the AnimationFeature. Mixamo is a great place to go for finding free to use animation content.

[roschler]

@c-morten Thanks again! I'll check it out. I hope it works better than the native Sumerian stuff though. I haven't gotten that work, either via state machine step using the PointOfInterest action set to the other Host, or from Scripting by emitting a ".setLookAtTarget" message on the system bus. The Host just seems to look around at random places, sometimes the other host's feet, etc.

[c-morten]

No problem, hopefully this repository can be helpful to you. Please don't hesitate to submit issues here if you run into errors or things don't work as expected.

[roschler]

@c-morten Thanks for all your help. I've got the HTML/three.js page working fine now, both animation and Polly speech. I'm surprised at how smooth the animation is and the little purple alien, is well, cute. :)