miralabs / mira-unity-sdk

Mira’s Unity-based SDK enables creators to develop original experiences, re-tool existing 3D assets, or add a premium layer to their existing handheld AR apps.
https://developer.mirareality.com
Other
53 stars 16 forks source link

Tracking multiple markers #11

Open araujokth opened 6 years ago

araujokth commented 6 years ago

I would like to track multiple markers in the same scene and I am having some difficulties understanding the tracking and hierarchy logic on your getting started documentation.

What I would like to do is to have one Mira marker to be tracked and define the origin of the coordinate system (as in your basic example, as I understood it), and then I would like to track another marker and just obtain the pose of my head relative to that second marker. What would be the proper way to achieve this?

Thanks for the help!

araujokth commented 6 years ago

Just an update. Been taking baby steps to try to understand how the marker tracking has been implemented so that I could try to track the second marker in the scene but I am having one issue with the Live Preview. I enabled "Concurrent targets = 2" in the "Trackers" gameobject in the WikiMiraTemplate, and I also did this in the LivePreview application.

I went to the script "MiraWikitudeManager.cs" and added some printouts to check when the CameraOverride function was running and if I could get the marker ID. I also added printout in the start function. When running in live preview, I get no printouts in the Unity console in the CameraOverride function, even though I can see that the marker is being tracked but I get a printout in the start function. However, when I run the example code without being in live preview, then I get printout both in the start and CameraOverride function in the Xcode console.

Any idea why the cameraoverride function is not being called OR why I am not getting these printouts in the console in Unity?

cbarronmira commented 6 years ago

Hi @araujokth , you may run into difficulties when using the live preview app in custom marker configurations - live preview is a sample scene with a single marker, and it streams the camera position and bluetooth controller data every frame from the phone to the editor. The editor receives this information, simulates the same behavior, and then sends the stereo camera feed back to the phone to be displayed. However, with multiple markers, certain events and data may not be currently set up to stream through these communication channels, and it may not give you the results you expect

araujokth commented 6 years ago

That makes sense and thanks for the explanation of the live preview. I was thinking of trying to define two separate markers in the live preview scene + app and hopefully getting it to work but perhaps the best is to first get it to work without live preview and then move to live preview.

But regarding the use of multiple markers, do you or anyone in the team have some advise about how I could achieve this? It would be great if you could just give me some pointers on new unity gameobjects and components I would have to add and some high level tips on the code to change and where. My plan was to try to be able to get some sort of marker ID for each marker being output in Unity so that I could edit the scripts to distinguish between the two types of markers; then one of the markers should be used in the same way you have developed for the single marker case, and the other I just wanted to get its pose.

Just another thing, what does the ""Concurrent targets" mean in the component in the Trackers gameobject?

Thanks again!

araujokth commented 6 years ago

Hi again @cbarronmira, I think there is some confusion between "targets" and "trackers" in this issue and #9. I am interested in having the tracking of multiple targets, which in the wikitude documentation says its possible if I am getting this correctly: https://www.wikitude.com/external/doc/documentation/7.2/unity/imagerecognitionnative.html. For that I have to set concurrent targets >1. However, I can only have one tracker enabled at a time, which I am assuming that it is OK to have 1 tracker for multiple targets.

I will keep reading this documentation and try out their tips.

araujokth commented 6 years ago

I managed to get the two markers to be tracked at the same time and confirmed this by doing printouts of the name of the marker detected inside the CameraOverride function in the MiraWikitudeManager.cs file. I was now trying to understand the camera position and rotation calculation you do inside the same function.

My goal is that if the Mira marker is detected, I let the camera position and rotation be calculated in the same way you have it now (so that the mira marker becomes the origin), but if the other marker is detected, I would like the camera position and rotation to NOT be updated, but instead, I just want to extract the pose of that marker relative to the camera pose. Any tips to understand how to interpret the poses available in this script would be greatly appreciated. Thanks!

araujokth commented 6 years ago

I now created a new object in the scene and in the CameraOverride function in the MiraWikitudeManager.cs file I am setting the position and rotation of this new object (its a public gameobject defined in the script which I added a cube prefab to the script in the editor under the MiraARCamera gameobject) as the position and rotation of "trackable", whenever my custom marker is visible. If the mira marker is visible then I just run the original code in the CameraOverride function. The original code is not run when my custom marker is tracked.

For some reason, the camera position and rotation still changes when my custom marker is tracked even though the original CameraOverride function code does not run. Is there any other place where the camera pose is updated according to the tracked marker? Or is this because of the hierarchy of my object in the Unity editor? I have tried to define the object under MiraWikiTracking, Trackers (and a new tracker) and MiraARCamera and nothing changes.