search

HL2-DINO / DINO-Unity

A Unity project for the HoloLens 2 to detect and track infrared markers / tools
https://hl2-dino.github.io/DINO-Unity/
BSD 3-Clause "New" or "Revised" License
15 stars 2 forks source link

Tracking Not Working with Custom Tool #4

Closed ThomasK-CADS closed 2 months ago

ThomasK-CADS commented 5 months ago

Hi!

I successfully deployed the Sample Project to my HoloLens 2, but the tracking is not working. The ToolDictionaryPrinter continuously prints an identity matrix.

I have double-checked all the setup steps, and the HoloLens' Research Mode is activated.

I used a custom tool with spherical markers instead of flat ones. Could this be causing the issue? I also tested my custom tool with the project referenced by Andreas Keller , and it worked without any problems.

Any suggestions would be greatly appreciated. Thank you!

hiqb217 commented 4 months ago

Hi!

The short answer is: no, using spherical markers currently isn't supported, and should in theory currently give you slightly wrong looking poses, but I would expect something to show up, and not just the identity matrix.

The longer answer:

It should be relatively easy to add some extra code to the DINO-DLL project to support spherical markers. It would involve reading and parsing the radius of the marker (e.g. from the config.json file that's passed from DINO-Unity to DINO-DLL), and accounting for the radius when handling depth values of the detected centre of each infrared marker.

Andreas's project does something similar I believe:

Snippet from IRToolTrack.cpp:

spheres_xyz.forEach(
    [&](cv::Vec3f& xyz, const int* position) -> void {
        xyz[2] = xyz[2] + cur_radius;
        cv::Vec3f temp_vec(xyz[0], xyz[1], 1);
        xyz = cv::Vec3f((temp_vec / cv::norm(temp_vec)) * xyz[2]);
    }
);

I'm leaving this as a note for myself or for any others who may be up for submitting a pull request to implement these changes to DINO-Unity and DINO-DLL!

bolebalazs commented 4 months ago

Hi @ThomasK-CADS, Were you able to implement the changes needed for spherical markers?

hiqb217 commented 2 months ago

@ThomasK-CADS @bolebalazs

Hey both. Please see this issue: https://github.com/HL2-DINO/DINO-Unity/issues/8 where I've provided info on an experimental update to track tools with spherical markers. Hope this helps!