ultraleap / UnityPlugin

Ultraleap SDK for Unity.
https://docs.ultraleap.com/unity-api/
Apache License 2.0
531 stars 168 forks source link

Fiducial Improvements #1631

Closed MattFiler closed 5 months ago

MattFiler commented 5 months ago

Summary

This MR adds the improvements I've worked on over the past couple days for TrackingMarkerObject to improve stability vs the current plugin implementation. Instead of just using the latest fiducial frame from the service as the truth for position, it's now a bit smarter and reconstructs the AprilTag frames Unity side. Using the reconstructed frames it figures out the position the object should be in by averaging the location of every tracked marker, with a bias towards the marker considered to have the least error from the platform (a value used at platform level and validated to be useful).

These changes have made a significant improvement to robustness in Barista - video can be found in the #sick_marker_demos channel!

Hopefully this is in a mergable state! I've left some debug visualisation stuff in as I feel it's useful to see what's happening under the hood, but feel free to remove it if you'd rather not have it there. If you select the object in scene you'll see:

Also if your markers have a child MeshRenderer they'll get updated to colours to represent their tracked state - green shows the one with bias, and orange shows tracked.

Contributor Tasks

Test Cycle

Link to the test cycle here.

Reviewer Tasks

Related JIRA Issues

If this MR closes any JIRA issues list them below in the form Closes PROJECT-#

Pull Request Templates

Switch template by going to preview and clicking the link - note it will not work if you've made any changes to the description.

You are currently using: default.md

Note: these links work by overwriting query parameters of the current url. If the current url contains any you may want to amend the url with &template=name.md instead of using the link. See query parameter docs for more information.

MattFiler commented 5 months ago

One thing that needs fixing: if an object is not at 0,0,0 in the scene the offsets will go weird.

Additionally there should probs be a check to make sure the transform the user sets as the target transform isn't the object itself or any of its children :)