Bbox labels in point cloud potentially incorrect

[jonasdieker]

Hi there,

first of all thank you for creating the devkit and open sourcing your data.

I visualized your data using the WebLabel Player and noticed that while the bounding boxes in the images seems extremely accurate and detailed, there are quite a few frames where the bounding boxes in the point cloud seem incorrect. Below is a picture to illustrate what I mean: The train bboxes are crossing several tracks and the switch (very light blue) at the bottom is almost perfectly perpendicular to the track.

Additionally, I tried to project the lidar data into the rgb_center camera and seem to have some strange scaling issue, which could well be due to a bug on my end. However, I noticed that the JSON file also provides LiDAR intrinsic parameters.

            "lidar": {
                "type": "lidar",
                "uri": "lidar",
                "stream_properties": {
                    "intrinsics_pinhole": {
                        "camera_matrix": [
                            1000,
                            0,
                            10,
                            0,
                            0,
                            1000,
                            10,
                            0,
                            0,
                            0,
                            1,
                            0
                        ],
                        "distortion_coeffs": [
                            0,
                            0,
                            0,
                            0,
                            0
                        ],
                        "width_px": 4112,
                        "height_px": 3008
                    }
                }
            }

Therefore I was wondering if an additional scaling is required to transform the lidar data into the correct units?

Thanks in advance!

[romantilly]

Thank you for your feedback!

Unfortunately, there are indeed some errors in the coordinates of the 3D bounding boxes in some of the sequences. We recognized that after publication and are working on a fix. The boxes were positioned correctly in the labeling tool but were not properly exported to the ASAM OpenLabel / RailLabel format. We will fix that and will re-upload the sequences as a new version as soon as possible.

As for the Lidar intrinsic parameters: As the Lidar coordinate system is the base / reference coordinate system for all other sensors, there should be no intrinsic parameters for the Lidar. That will be corrected as well.

[jonasdieker]

Thank you for your reply! Looking forward to the updated version. Any rough idea (weeks, months) when this will be released?

[egemenkopuz]

@jonasdieker I am also having a hard time trying to project lidar data into rgb data such as "rgb_center". My projected clouds points ( or 3d bounding boxes) on an image frame are off by moderate margins with weird scaling issue like you experienced. Have you been able to identify the reason behind this problem?

[jonasdieker]

@egemenkopuz I have not tried again since I opened the issue. I will let you know when I have made progress!

[jonasdieker]

Hi @romantilly, have the labels been fixed yet? I took a brief look at the changelog.md and it does not seem like it.

[romantilly]

Hi. We are working on the update. I cannot promise a specific date, but my estimate is that the update will be online in only a few weeks.

[romantilly]

The update has been published under the same DOI.

We have also published the research report for the project and the labeling guide that describes the rules underlying the annotations in OSDaR23.

[jonasdieker]

That is good news, thank you for letting me know!