New 'Character Model' augmentation #177

Closed wekitecs closed 2 years ago

In GitLab by @fwild on Jan 7, 2021, 11:52

Let's create a new augmentation (similar to 'Ghost'), where:

we can select a 3D character model (from a selection, including the PBIS alien)
we can checkbox activate behaviours (like 'path finder' or 'random walk' or 'try to bump onto user')
using the same animation files for all of them ('idle', 'walk', ...)
?? should we have an input field for 'how many' so that tracking the number of 'bumps' the user did with any of them becomes more easy?
?? how do we display the number of bumps? Or are they evaluated in the next action step? Maybe requesting the number of bumps from the xAPI endpoint?

In GitLab by @fwild on Jan 7, 2021, 11:54

Maybe create another augmentation (to be used in the second action step) that can gather data from the xAPI about where the user walked, where the user bumped into a virtual character model?

In GitLab by @wild on Feb 2, 2021, 16:59

We should put a ground plane in automatically - to avoid that the character falls through the floor? Here is a tutorial for the old HoloToolkit: https://codereality.net/ar-for-eu-book/chapter/development/tools/unity/mrtk/mapgazegesture/

The relevant line is the one about: horizontal = SurfaceMeshesToPlanes.Instance.GetActivePlanes(PlaneTypes.Table | PlaneTypes.Floor);

While the HoloToolkit (MRTk) has changed in version 2, I'd hope it still is very similar and the access methods have not changed much.

In GitLab by @wild on Feb 2, 2021, 17:00

We should use physics - to make sure that the characters are running on the mesh of the floor...

In GitLab by @william.guest on Feb 3, 2021, 10:47

unfortunately the SurfaceMeshesToPlanes object seems to be not yet implemented in MRTK. I am not sure if it is up-to-date, but this link:

https://microsoft.github.io/MixedRealityToolkit-Unity/Documentation/HTKToMRTKPortingGuide.html

shows the following:

In GitLab by @william.guest on Feb 3, 2021, 10:48

I have extracted those parts of the HTK that can perform these funtions and could add them to the project, if we agree that this is a good plan.

In GitLab by @robhillman97 on Feb 3, 2021, 11:07

Just ran a quick test on this and it looks like as long as an object has a RigidBody component attached it wont fall through the spatial map. I haven't tested this on a HoloLens yet though.

Gravity

In GitLab by @william.guest on Feb 3, 2021, 11:08

i think the issue we ran into was when the size of the object was roughly the same as the mesh polygon. it looks like the mesh is more dense now and it's possible they have also sealed it a little better.

In GitLab by @wild on Feb 3, 2021, 13:58

In the past (with ATLAS) we often had the problem that small objects fall through the cracks of the mesh - and objects fall into the void, where no mesh has been scanned yet... not sure this is prevented now?

In GitLab by @wild on Feb 4, 2021, 13:02

One behaviour could be "teachable agent": The kids can control the 3D character model (e.g. the ARETE alien) by (air-)tapping to make it walk to a target location (or via voice command “let’s go”). As an alternative to the ones listed in the ticket: 'path finder' (that's the same as 'random walk', right?) or 'try to bump onto user' - that's a more aggressive version of random walk).

In GitLab by @wild on Feb 18, 2021, 11:43

@robhillman97 what is this new augmentation currently called? Can we go for 'character' to keep it short and crisp?

In GitLab by @robhillman97 on Feb 18, 2021, 11:46

I've just been using "Character Aug" as a place holder so "Character" works well

In GitLab by @wild on Feb 26, 2021, 14:43

What is the number field for? "accept" does not work for me - the config panel of the augmentation on the task card does not disappear. It still does "accept" the job, though - as ghosts keep running around...

In GitLab by @wild on Feb 26, 2021, 14:44

The ghosts seem to float? Do they walk over the spatial map plane?

In GitLab by @robhillman97 on Feb 26, 2021, 15:29

The number field sets the number of characters that are instantiated when accept is clicked. I have also now realised that the augmentation is not integrated correctly as clicking the accept button does not close the config panel. As well as this the augmentation isn't added to the action list and therefore cant be edited or deleted once it is created. At the moment the characters move to an invisible object that rests on the spatial map plane, I did this to prevent the characters falling through the floor as was happening when I was testing. I think I need to look more into SurfaceMeshesToPlanes for MRTK in order for them to truly walk along the floor of the spatial map. As for the floating I assumed the ghost model was just the torso and head so added a gap between the floor and the body where the legs would be if I used a different model.

There is still a lot to do for this issue, starting I think by making sure that the augmentation is added correctly to the action list. Then I will start improving the functionality of the augmentation and making the characters move correctly on the spatial map and then tackling the different character models.

In GitLab by @wild on Feb 26, 2021, 16:33

I think we do not need a number of characters - if you want multiple, you should add multiple character augmentations. :)

In GitLab by @wild on Feb 26, 2021, 16:33

I do realise now that it drops an invisible cube on the floor to set the ground level. Would be good to make that somehow visible? I tried the feature always sitting in a chair - so it must have fallen on my lap. That explains the behaviour :)

In GitLab by @wild on Feb 26, 2021, 16:34

The character models are not attached as children of the task station - that's why they are not removed with the next action step.

In GitLab by @wild on Feb 26, 2021, 16:34

Can we use Alla or Hanna instead of the ghost? With a standard walk animation? Xinyu can provide that if you don't have it yet?

In GitLab by @wild on Feb 26, 2021, 16:35

And the accept button does not yet work - it does create the augmentations, but it does not close the dialogue of adding. That must be a quick fix, I'd think. Once accepted, it should simply show up in the list of augmentations attached on the task card.

In GitLab by @wild on Feb 26, 2021, 16:36

Can we fix all these? I think they are quick things - and we then could include this feature in the Sprint 5 release :)

In GitLab by @wild on Feb 26, 2021, 16:36

(Oh: the ARLEM code does not yet work seemingly? That may require bit more thinking - but should copy the ARLEM style of the ghost...)

In GitLab by @robhillman97 on Feb 26, 2021, 17:01

I can get working on fixing some of these and swapping in a different model. When it comes to ARLEM side of things I might have to ask somebody to give me a crash course on setting them up for new augmentations and I can get that working for sprint 6.

In GitLab by @william.guest on Feb 28, 2021, 13:09

happy to help out with this, moving the ticket to sprint 6.

@john.murphy2 i think this would be helpful for you too. let's meet soon to go over this.

In GitLab by @a85jafari on Mar 8, 2021, 24:50

unassigned @robhillman97

In GitLab by @a85jafari on Mar 8, 2021, 24:51

Following Fridolin email, I will continue on this issue if it is OK

In GitLab by @a85jafari on Mar 9, 2021, 19:11

Fixed !161

In GitLab by @a85jafari on Mar 9, 2021, 19:11

closed

WEKIT-ECS / MIRAGE-XR

New 'Character Model' augmentation #177