Several questions on extension to a ROS node for localization

slensgra commented 4 years ago

First off, thank you for releasing this code base. I'm evaluating using STags for a mobile robot application where the stability of the reading from the tags is extremely important.

Pose estimation code?

I've got a simple ROS integration started here: https://github.com/dartmouthrobotics/stag. I'm starting out with finding the pose of the tag using opencv's solvePnP function with a hard coded camera and distortion configuration (used here https://github.com/dartmouthrobotics/stag/blob/master/src/Stag.cpp#L89). In your paper you mention using a more recent algorithm for finding the camera extrinsics with the refined homography matrix but as far as I can tell you don't provide your implementation of that algorithm. Do you have the implementation you used for your experiments handy? If so, I'd appreciate getting a copy!

Support for configurable tag sizes?

Would you be open to adding configuration for a mapping of tag id to tag size? If so, I'd be happy to look into doing that. Insight into the places that need to be changed in the code would help that go faster.

Marker decoded many times in single frames

When the marker is successfully decoded, the array of decoded markers will usually contain four duplicate readings for the same tag which I'm sure has performance implications. For example, in this frame 4 quads the marker is decoded four times (the array markers on the Stag object has four markers in it here https://github.com/dartmouthrobotics/stag/blob/master/src/Stag.cpp#L50). I would guess that this is coming from the quad detector and that adding a simple step which filters out duplicate quads based on the sum of the distances between their corners or something would fix this. Output from that frame:

~~~~~~~~~~~~~~~~~
Marker 1 num marker 4
Center [970.951, 268.624]
Translation: [-0.5426630776763407;
 -1.027159407186172;
 2.694480358554943]
Rotation: [-0.0444761597568246;
 0.1080291508322071;
 0.01037789328024481]
Marker 1 num marker 4
Center [970.955, 268.627]
Translation: [-0.5433152177014651;
 -1.026926771242593;
 2.693072105601706]
Rotation: [-0.04121157508080839;
 0.1051655381738611;
 0.009573907338546446]
Marker 1 num marker 4
Center [970.951, 268.624]
Translation: [-0.5426630776763407;
 -1.027159407186172;
 2.694480358554943]
Rotation: [-0.0444761597568246;
 0.1080291508322071;
 0.01037789328024481]
Marker 1 num marker 4
Center [970.955, 268.627]
Translation: [-0.5433152177014651;
 -1.026926771242593;
 2.693072105601706]
Rotation: [-0.04121157508080839;
 0.1051655381738611;
 0.009573907338546446]

bbenligiray commented 4 years ago

Pose estimation code

The refined corners are fed into this pose estimation method because it is more robust against the ambiguity in pose from 4 planar points compared to solvePnP(). https://github.com/paroj/artoolkitplus/blob/afcc4cd7b79bb27879a066f5c2dc3ac58e371182/src/core/rppGetTransMat.cpp If you are using multiple markers to estimate the pose, I'd just use solvePnP().

Support for configurable tag sizes

If I understand correctly, this is about ID#1 being 10cm x 10cm and ID#2 20cm x 20cm, etc. If so, this sounds too use case-specific for this project.

Marker decoded many times in single frames

My co-author pointed this out to me recently, this is indeed a serious problem. This bug didn't exist in the experiments for the paper, but I introduced it in the refactoring after. Originally, the algorithm is supposed to stop searching for markers on an edge segment as soon as it has found one (and validated with decoding). Unfortunately, I'll only be able to fix this in a month.

slensgra commented 4 years ago

Thank you for getting back to me.

I'll add in a little bit of logic to ignore markers which have already been decoded for now and will handle the size configuration from my end.

Brenn10 commented 4 years ago

Hi all. If you would like, we have recently been working on a STag ROS implementation as well. We allow for configurable tag sizes and bundles as well. If you would like to use it, you can find it here: https://github.com/usrl-uofsc/stag_ros/

slensgra commented 3 years ago

And here is the one I eventually made that has these features as well :) https://github.com/dartmouthrobotics/stag_ros

Brenn10 commented 3 years ago

Trying to steal our thunder? :)

bbenligiray commented 3 years ago

Hey @slensgra , thanks for the contribution! I linked your repo in the README.

Leaving the issue open for documentation purposes.

slensgra commented 3 years ago

Trying to steal our thunder? :)

I think we worked past each other on this! I started mine right after I made this issue but didn't set up the README for being searchable. We do a some work with the AFRL over at UofSC if you know them. When we finally publish, I'll cite your paper on the stability experiments

bbenligiray / stag