kit_tray tape interfere with Aruco detection?

[AravindaDP]

Apologies for reporting this late. I thought I had reported this earlier but just now realized I have missed it.

We are using cv2.aruco.ArucoDetector with default cv2.aruco.DetectorParameters and noticed that sometimes cv2.aruco.ArucoDetector.detectMarkers(cv_image) fails to detect tray markers (cv_image is constructed using CvBridge.imgmsg_to_cv2(img_msg, desired_encoding="mono8"))

When we made the tape material less darker by setting ambient and diffuse values as below at https://github.com/usnistgov/ARIAC/blob/ariac2024/ariac_gazebo/models/kit_tray/model.sdf#L39 it started working consistently. (perhaps it could be little bit more darker but we are happy with these values)

      <visual name="tape">
        <geometry>
          <mesh>
            <uri>model://kit_tray/meshes/tape.stl</uri>
          </mesh>
        </geometry>
        <material>
          <ambient>0.4 0.4 0.4 1.0</ambient>
          <diffuse>0.4 0.4 0.4 1.0</diffuse>
        </material>
      </visual>

I'm not sure it's too late to ask such a change. But appreciate if it could be considered. (Alternatively we would welcome if it can be solved by applying non-default parameters in cv2.aruco detecters as well as a last resort.)

Approx additional troubleshooting time occurred due to the issue during competition : ~0.5 days

[dan9thsense]

Can you explain the change that was made? Our tray detection has been working OK and if you change the sdf now, we have to restart a lot of testing with very little time before finals deadline. What happens if this change makes our model no longer reliably work?

[jaybrecht]

The color of the tape was change to be gray instead of black. If this causes an issue with your system please let me know and we will figure out a solution.

[dan9thsense]

Was that incorporated into the recent pull request that fixed the flip issue? If so, then we're good, as I used that branch for testing over the weekend and it was ok.

Get Outlook for Androidhttps://aka.ms/AAb9ysg

---

From: Justin Albrecht @.> Sent: Tuesday, May 28, 2024 2:50:40 PM To: usnistgov/ARIAC @.> Cc: Dan @.>; Comment @.> Subject: Re: [usnistgov/ARIAC] kit_tray tape interfere with Aruco detection? (Issue #352)

The color of the tape was change to be gray instead of black. If this causes an issue with your system please let me know and we will figure out a solution.

Screenshot.from.2024-05-28.14-47-56.png (view on web)https://github.com/usnistgov/ARIAC/assets/50928580/2bdfb941-5938-45d1-b62e-3efef9901a8c

— Reply to this email directly, view it on GitHubhttps://github.com/usnistgov/ARIAC/issues/352#issuecomment-2135904867, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AAH6RL3BGG7P26KPI26DANDZETGYBAVCNFSM6AAAAABIH3SIVSVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCMZVHEYDIOBWG4. You are receiving this because you commented.Message ID: @.***>

[jaybrecht]

No, I just merged that PR this morning. The branch 2024.5.0-staging currently has all new changes that will be included in the 5.0 release.

[saahu27]

The detection using cv2.aruco.ArucoDetector can sometimes fail if you pass in the entire image of the kittray. You can alternatives mask the image with hardcoded pixel coordinates to crop out only the aruco tag and pass that image to opencv aruco detector. This is a better option to detect the Aruco tags.

[AravindaDP]

Yes, Currently we are testing that approach using the assumption that kit trays are at predesignated coordinates for this year competition. We also noticed that while original color values for kit tray tape material infact wasn't pure black (but still dark enough to confuse ArucoDetector) so color threshold based filtering also should be possible.

So since now deadline is pushed back if other teams have objections to changing kit tray tape color we would be able to adopt one of above alternatives.