Embedded PointOfInterest commands are not working?

[roschler]

I am using the following text on the Sumerian test page:

<speak>This is a bad day at the shoe <mark name='{"feature":"PointOfInterestFeature","method":"setTargetByName","args":["camera"]}' /> counting department.</speak>

When I click the Play button, Luke looks off into space several clicks to the right of the camera, not at the camera itself. However, if I use this Javascript code:

host.PointOfInterestFeature.setTarget('camera');

It works fine. What is wrong with my embedded mark command?

I also tried same command with Luke to get him to look at the Alien character using :

<speak>This is a bad day at the shoe <mark name='{"feature":"PointOfInterestFeature","method":"setTargetByName","args":["Alien"]}' /> counting department.</speak>

But again, he stares off into space at the same place as the command with the "camera" as the target. What is wrong with my embedded PointOfInterestFeature command targets? What is the correct syntax to look at the "camera" or one of the other hosts?

[roschler]

I believe I know what the problem is.

Note to all people coming from the "main" AWS/Amazon Sumerian environment over to this repo. The Amazon Sumerian host names will not help you when it comes to locating object's in the ThreeJS/Babylon animation environment. They are not the same thing. If you are used to using the find-component method in the "main" Amazon Sumerian then you will be confused when you get to the ThreeJS/Babylon environment.

As the ThreeJS docs say:

https://threejs.org/docs/#api/en/core/Object3D.getObjectByName

".getObjectByName ( name : String ) : Object3D name -- String to match to the children's Object3D.name property.

Searches through an object and its children, starting with the object itself, and returns the first with a matching name. Note that for most objects the name is an empty string by default. You will have to set it manually to make use of this method."

So apparently this repo's code is setting the names for various child objects when they are constructed . By tracing through the main sample page code for this repo I found this code, which you will find in the createHost() function for the sample page (Note, the comments are mine, the code is from the page):

            /**
             * ROS: This is a classic case of type morphing.  config.reference
             *  starts out with a string label that represents the "LookHead"
             *  target and ends up BEING the object that has that ThreeJS
             *  name.  I assume it's a ThreeJS object because getObjectByName()
             *  is a ThreeJS function.  (Note: the docs say that normally an
             *  object's name is an empty string so the second assumption is
             *  that the Sumerian Hosts code or some other code is creating
             *  the value for that property that is being used as the lookup
             *  string here.)
             */
            // Find and store reference objects
            config.reference =
                character.getObjectByName(
                                        config.reference.replace(':', '')

During tracing you will see that some code from this repo placed the joint/object name char:def_c_neckB into the config.reference property for the character being built. The code underneath my comment looks up the ThreeJS/Babylon object with that name and replaces that string value in config.reference with a JS object reference to the actual animation object that label was assigned to, after removing the colon embedded in that joint/object name. In other words, that code replaces the animation object name with the appropriate object reference.

@c_morten How can I build a name that, when passed to character.getObjectByName() function, will resolve to the desired "look-at" joint in another character (i.e. - not the same character as the host character whose head I want to turn)?

For example, given the character Luke, how could I craft a target name that will resolve to the correct look-at target on the Alien character so that Luke looks at Alien, when using the PointOfInterestFeature embedded JSON notation/feature? What value similar to char:def_c_neckB will have that desired effect?

UPDATE: I realize the above context has a problem because the getObjectByName() function is attached to the character/host reference. Is there a "global" getObjectByName() function that cuts across character/host boundaries, similar to the find-by-name entities function in the "main" AWS Sumerian platform that can search the entire world object space? If not, how can I use the "" feature to do the equivalent?

[c-morten]

Hi @roschler. All of the names in the host character hierarchies are determined by the contents of the character glTF files, we do not manually manipulate any of these names after import. three.js handles all of the import and naming logic. The only caveat is that we used namespaces in our character assets (all object names are prefixed by the string "char:"), and three.js strips the ":" character from names on import.

The reason your host seems to look off in the wrong direction with some of your setTargetByName commands is that you are either passing a name that does not exist in the scene or you have not provided a three.js Scene object to the PointOfInterestFeature when you add it. The Scene object is necessary for using this method because that is the parent object we search under when using the THREE.Object3d.getObjectByName method. In this situation you are effectively setting the point of interest target to undefined, it does not know what to look at so it is reverting to the default direction of where the character asset was looking at the time of animation authoring. This is the line of code where we pass the scene object to the poi feature in the three.js example: https://github.com/aws-samples/amazon-sumerian-hosts/blob/mainline/examples/three.html#L855. In your sample SSML markup, you are trying to look at an object named "Alien". Although we refer to one of the hosts as "Alien" in some of the code, we have not named any nodes "Alien" so it that's still not a valid point of interest target. setTargetByName can be a bit limiting because you can have many nodes in the scene with the same name, and this is actually the case for the non-alien hosts. You can refer to the "Nodes" object in the glTF files for a complete list of node names inside each file. A workaround you can use if you have 2 hosts with the same node names would be to create a uniquely named THREE.Object3d for each host and parent it to the node you want to target when you import the assets. Then you can refer to that name in your speech SSML. A second way to make this work would be to use setTargetById rather than setTargetByName. I think this is the more cumbersome of the 2 solutions because object IDs are generated on import, so you wouldn't be able to determine your speech text beforehand. With the node creation technique you can use whatever names you'd like during speech text creation and then create nodes to match after your import process.