Using AprilTags returns error

[mgagvani]

When I add an AprilTags JSON file to the config I get this error:

[184430103199860E00] [1.1] [1.990] [Script(9)] [critical] AttributeError: 'lpb.ImgFrame' object has no attribute 'getTimestampDevice'

At:
  <script>(10): <module>

F: [global] [         0] [ThreadN] dispatcherResponseServe:925  no request for this response: XLINK_READ_REL_RESP 1

If I comment out the config.aprilTagPath part the code will run.

I am using an Oak-D Pro camera. My JSON configuration file is below:

[
    {
    "id": 0,
    "size": 0.152,
    "family": "tag16h5",
    "tagToWorld": [
        [0, 0, 1, 0],
        [0, 1, 0, 1.494],
        [-1, 0, 0, -0.321],
        [0, 0, 0, 1]]
    },
    {
        "id": 1,
        "size": 0.152,
        "family": "tag16h5",
        "tagToWorld": [
            [1, 0, 0, 1.735],
            [0, 1, 0, 2.058],
            [0, 0, 1, -0.435],
            [0, 0, 0, 1]]
    }
]

Any help would be appreciated.

[kaatrasa]

Hey,

It doesn't look like the error messages should be related to AprilTags, but if setting aprilTagPath is causing some issues, then it would be good to verify that your apriltag config file is correct. I would double-check the tagToWorld transformations that are a little tricky to get right.

For instance, is your second tag attached to the ceiling? We use z-is-up world coordinate system (z = -gravity), and now you have tag.z == world.z. To get started, you could try with a single AprilTag whose y-axis is pointing towards gravity direction and use this transformation:

"tagToWorld": [
[1, 0, 0, 0],
[0, 0, 1, 0],
[0, -1, 0, 0],
[0, 0, 0, 1]]

These resources might be useful: https://github.com/SpectacularAI/sdk-examples/tree/main/python/oak#coordinate-systems https://github.com/SpectacularAI/docs/blob/main/pdf/april_tag_instructions.pdf

If the issues persist, then maybe the easiest option would be for you to share a recording with us, see vio_record.py.

[mgagvani]

Thank you. The tag to world transforms are a bit confusing to me. I see the example you gave in your comment and in the apriltag instructions PDF but I don't understand where I am supposed to input the actual ground truth position of the AprilTags in this. My tagToWorld looks like this:

"tagToWorld": [
        [0, 0, 1, 0],
        [0, 1, 0, 1.494],
        [-1, 0, 0, -0.321],
        [0, 0, 0, 1]]

This apriltag is pointed horizontally on a wall, and from my understanding, since Z goes into the AprilTag, I flipped that number in the matrix. And then the right-hand column is the x,y,z translation of the apriltag from the origin. If the configuration is like how you specified above where should I be putting the actual position of the apriltag?

[kaatrasa]

tagToWorld is a transformation matrix like this:

$$\begin{bmatrix} \mathbf{R} & \mathbf{t} \ \mathbf{0}^T & 1 \end{bmatrix}$$

where $\mathbf{R}$ is a 3x3 rotation matrix and

$$ \mathbf{t} = \begin{bmatrix} tx \ ty \ tz \end{bmatrix}$$

is 3x1 translation vector, which defines the tag's position in world coordinate frame. The rotation part is tricky to get right without visualization (we will try to add one eventually). If you are unsure about it, I would suggest you place the tag on a wall, keep its original orientation and use the rotation matrix

$$ \mathbf{R} = \begin{bmatrix} 1 && 0 && 0 \ 0 && 0 && 1 \ 0 && -1 && 0 \end{bmatrix}$$

If you have more AprilTags then you could try virtually placing the tags in some 3D visualization (e.g. Blender), and exporting the transformations from there.

[mgagvani]

Thanks for your input @kaatrasa. I understand the tagToWorld transformation but I am receiving a low-level error from the camera whenever I add this to my code:

config = spectacularAI.depthai.Configuration()
config.aprilTagPath = "test_apriltags.json"
pipeline = depthai.Pipeline()
vio_pipeline = spectacularAI.depthai.Pipeline(pipeline, config)

The error is the original issue:

[184430103199860E00] [1.1] [1.990] [Script(9)] [critical] AttributeError: 'lpb.ImgFrame' object has no attribute 'getTimestampDevice'

At:

Githubissues.

Githubissues is a development platform for aggregating issues.