Charuco detections not working on agrob bag file

[aaguiar96]

Hi @eupedrosa

We already have the labeler for the velodyne PCL. While generating the .json annotations, we noticed that the charuco is not being detected. Can you help us understand the reason?

Thank's in advance. :)

[eupedrosa]

Yes, I can help.

The problem is here: https://github.com/lardemua/AtlasCarCalibration/blob/655a91859344afdb50d65c2b7b2abbe50c8c16a9/interactive_calibration/src/interactive_calibration/patterns.py#L45

I am 100% certain the dictionary you used is not this one. Change the dictionary to DICT_4X4_50.

[miguelriemoliveira]

Hi,

I suggest to have the dict type in the config.json.

Can tou do it André?

On Fri, May 1, 2020, 19:13 Eurico F. Pedrosa notifications@github.com wrote:

Yes, I can help.

The problem is here: https://github.com/lardemua/AtlasCarCalibration/blob/655a91859344afdb50d65c2b7b2abbe50c8c16a9/interactive_calibration/src/interactive_calibration/patterns.py#L45

I am 100% certain the dictionary you used is not this one. Change the dictionary to DICT_4X4_50.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/lardemua/AtlasCarCalibration/issues/146#issuecomment-622500024, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACWTHVSUDFFFWOBN6I4H7ALRPMGLFANCNFSM4MXG63NQ .

[miguelriemoliveira]

Thanks @eupedrosa .

@aaguiar96 , if this works, can you add a parameter on the config.json for the selection of the charuco type?

[aaguiar96]

So, I changed the dictionary to DICT_4x4_50 but I'm getting the following error:

error: OpenCV(4.2.0) /io/opencv_contrib/modules/aruco/src/dictionary.cpp:141: error: (-215:Assertion failed) id >= 0 && id < bytesList.rows in function 'getDistanceToId'

After that, I tried DICT_4x4_250 but did not obtain any detection with this dictionary, even when the chessboard is really close to the camera...

[miguelriemoliveira]

Hi @aaguiar96 ,

take a look at this

https://stackoverflow.com/questions/53269495/error-215assertion-failed-id-byteslist-rows-in-function-drawmarker

(the suggestion is also to use the 4x4x250, but perhaps there's more there)

do you know which dictionary you used to produce your printable charuco? How did you do it?

[aaguiar96]

I've already seen that issue, that's why I changed to DICT_4x4_250. What I understand is that the error is related with the number of detections. Increasing the limit to 250, the error disappears. What is strange is that no detections are found.

Like @eupedrosa said, I'm pretty sure that I used DICT_4x4 - this one.

[aaguiar96]

Ok, perhaps it could be related with the Charuco parameters one the config.json. Does the detector use the pattern_size, border_size, dimension, etc?

[miguelriemoliveira]

yes it does. Check here

https://github.com/lardemua/AtlasCarCalibration/blob/655a91859344afdb50d65c2b7b2abbe50c8c16a9/interactive_calibration/src/interactive_calibration/patterns.py#L40-L72

Those numbers should be well configured

[aaguiar96]

Yes, you're right. It might be it... I'm trying to refine the dimensions to see if I can get detections.

[aaguiar96]

@eupedrosa can you tell me what is the difference between the size and the inner_size parameters? Is the inner_size being used?

[eupedrosa]

The size is the checker width and the inner_size is the width of the aruco marker (usally 5mm smaller than checker width).

[aaguiar96]

Ok, these are my settings for agrob:

    "calibration_pattern" : {
        "link": "chessboard_link",
        "parent_link": "base_link",
        "origin": [-0.37, 0.0, 0.0, 0, 0.0, 3.8],
        "fixed": true,
        "pattern_type": "charuco",
        "border_size": 0.12,
        "dimension": {"x": 11, "y": 8},
        "size": 0.029,
        "inner_size": 0.024
    }

I think they are quite close to the real values (measured it in rviz), but there might be some errors. Could these errors lead to the non detection?

[eupedrosa]

Did you try? Did you change the dictionary to DICT_4x4_50?

This has nothing to do with the detection, but for your use-case fixed should be false.

[aaguiar96]

Yes, I'm using DICT_4X4_50. I've made some debug, and the array rejected from the aruco detection has a lot of points.

This has nothing to do with the detection, but for your use-case fixed should be false.

Ok. :)

[eupedrosa]

Use the view_pattern.py script to quick iterate the debug process.

[aaguiar96]

With that script, I can see detections in a few frames (1%) when the chessboard is close to the camera. It's strange because there are very frames where the chessboard is quite noticeable but no detections are found.

[eupedrosa]

Can you show an image where it detects and one where not?

[aaguiar96]

Here:

[eupedrosa]

The images are to dark and the detection is on black & white. Maybe some pre-processing? @miguelriemoliveira you are the expert here, in this type of things.

[aaguiar96]

Ok, that makes sense. @eupedrosa I also noticed that the detections only take place at the first few frames. Coincidence right?

[eupedrosa]

@eupedrosa I also noticed that the detections only take place at the first few frames. Coincidence right?

Most likely.

[miguelriemoliveira]

How to make the images brighter? Perhaps an histogram stretch would help ...

I will runsome tests on the view script.

André can you push the latest well configured json?

On Mon, May 4, 2020, 11:11 Eurico F. Pedrosa notifications@github.com wrote:

@eupedrosa https://github.com/eupedrosa I also noticed that the detections only take place at the first few frames. Coincidence right?

Most likely.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/lardemua/AtlasCarCalibration/issues/146#issuecomment-623377365, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACWTHVXPYWXGDHLMHFMER43RP2IE5ANCNFSM4MXG63NQ .

[aaguiar96]

André can you push the latest well configured json?

Done @miguelriemoliveira :)

[miguelriemoliveira]

Thanks, will get back to you.

[miguelriemoliveira]

Hi @aaguiar96 ,

I cannot see a single detection ... am I running this right?

rosrun interactive_calibration view_pattern.py -t charuco -x 13 -y 8 -L 0.029 -l 0.021 /d/zed_node/left/image

[aaguiar96]

@miguelriemoliveira, I only saw one or two detections at the beginning of the bag (first two seconds) with the command:

rosrun interactive_calibration view_pattern.py -t charuco -x 11 -y 8 -L 0.029 -l 0.024 /zed/zed_node/left/image

[miguelriemoliveira]

Ah, so the config.json is incorrect (x=13). That could be the problem. Let me check.

[aaguiar96]

The config.json already considers x=11.

[miguelriemoliveira]

ups, forgot to pull ... my bad :). Which bag file are you using?

I am working with

agrob_2020-03-12-10-09-49_1.bag

and never see the pattern as close as you show in the images above. You are always further away ...

[aaguiar96]

I'm using calibration_2020-03-12-10-09-49_1.bag too. From seconds 0 to 10 I am close to the camera. In the first two seconds, I am quite close. :)

[miguelriemoliveira]

Hi again,

I found the portion of the bag you mention. I did some stuff on the launch files. You can run

roslaunch interactive_calibration agrob_calibration.launch bag:=/home/mike/bagfiles/agrob/agrob_2020-03-12-10-09-49_1.bag bag_duration:=4

To playback only the first 4 secs.

Then I run

rosrun interactive_calibration view_pattern.py -t charuco -x 11 -y 8 -L 0.029 -l 0.024 /zed/zed_node/left/image

which I changed to stretch the histogram. The image is clearly brighter:

but I can see no detection ... not sure if I am running correctly. I never saw a single detection, even before stretching the histogram.

@aaguiar96 , can you pull an try it out?

BTW, don't forget to pull all, including agrob. I say this because many people forget to pull : )

[aaguiar96]

Now I cannot see any detection either... The image now looks really good. It's hard to understand why no detections are found.

@eupedrosa can you try it out? Or maybe, if you want, we can meet to talk about this. :)

[miguelriemoliveira]

I can meet around 14h

On Mon, May 4, 2020, 12:42 André Aguiar notifications@github.com wrote:

Now I cannot see any detection either... The image now looks really good. Is hard to understand why no detections are found.

@eupedrosa https://github.com/eupedrosa can you try it out? Or maybe, if you want, we can meet to talk about this. :)

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/lardemua/AtlasCarCalibration/issues/146#issuecomment-623414539, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACWTHVSUNCUWM5AWH3SH3JTRP2S3JANCNFSM4MXG63NQ .

[aaguiar96]

I have team meeting at 14h. Can you meet later? 14h45/15h?

[eupedrosa]

I have one at 14h and another at 15h. What about around 16h30?

[miguelriemoliveira]

14h45 ok. Call me once you are ready

On Mon, May 4, 2020, 12:47 André Aguiar notifications@github.com wrote:

I have team meeting at 14h. Can you meet later? 14h45/15h?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/lardemua/AtlasCarCalibration/issues/146#issuecomment-623416543, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACWTHVWDF25A2QIMJA3AVVDRP2TOJANCNFSM4MXG63NQ .

[miguelriemoliveira]

I have one at 16h15 ...

On Mon, May 4, 2020, 12:52 Eurico F. Pedrosa notifications@github.com wrote:

I have one at 14h and another at 15h. What about around 16h30?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/lardemua/AtlasCarCalibration/issues/146#issuecomment-623418363, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACWTHVU4MM4QCDSC4HI4UTTRP2UAFANCNFSM4MXG63NQ .

[eupedrosa]

By the way, you are using the wrong numbers, it should be rosrun interactive_calibration view_pattern.py -t charuco -x 10 -y 7 -L 0.029 -l 0.024

[aaguiar96]

I can meet anytime after 14h45 and before 19h30! :)

By the way, you are using the wrong numbers, it should be rosrun interactive_calibration view_pattern.py -t charuco -x 10 -y 7 -L 0.029 -l 0.024

Ok, actually that helped. Now it detects the chessboard when it is close to the camera. @miguelriemoliveira have you tried it out? Are those detections enough for testing the optimization?

[aaguiar96]

@miguelriemoliveira and @eupedrosa I added the dictionary option to config.json file. The default value for the charuco dictionary is DICT_5X5_100 - the one you were using.

To run with the view_pattern.py script, the following input as to be passed: -d DICT_4X4_50.

[miguelriemoliveira]

Hi @aaguiar96 ,

I have a code in which I had to configure the aruco detector's parameters to detect the arucos.

https://github.com/miguelriemoliveira/OCDatasetLoader/blob/858e7d9f5ffe8dc2efb071982fecf2bf691f645f/OCDatasetLoader/OCArucoDetector.py#L96-L110

I think the problem is that the arucos are too small, but perhaps experimenting with these values we could get a better detection

Here's a list of all the parameters

https://docs.opencv.org/trunk/d1/dcd/structcv_1_1aruco_1_1DetectorParameters.html

I run

roslaunch interactive_calibration agrob_calibration.launch bag:=/home/mike/bagfiles/agrob/agrob_2020-03-12-10-09-49_1.bag bag_duration:=7

and

rosrun interactive_calibration view_pattern.py -t charuco -x 10 -y 7 -L 0.029 -l 0.024 -d DICT_4X4_50 /zed/zed_node/left/image

and after second 3 I cannot get any detection. Also, try to relax this

https://github.com/lardemua/AtlasCarCalibration/blob/248f181a093c571c5c6edf87db71ac6ed824c8de/interactive_calibration/src/interactive_calibration/patterns.py#L99

So can you try changing the params to see if we can get a bit better detection? I could take some time.

Since I will be busy tomorrow and wednesday, perhaps we can speak now so I introduce you to the part about creating a dataset and adding your code in the objective function.

Can

[aaguiar96]

Thanks @miguelriemoliveira, I'll try it out.

I'm still on a meeting. I need more 15 minutes. Can you meet at 15h?

[miguelriemoliveira]

15h ok.

See you.

• Miguel Riem de Oliveira email: mriem@ua.pt | m.riem.oliveira@gmail.com office: 22.2.18 | ext: 23887 Dep. de Engenharia Mecânica, Universidade de Aveiro Campus Universitário de Santiago, 3810-193 Portugal*

On Mon, 4 May 2020 at 14:42, André Aguiar notifications@github.com wrote:

Thanks @miguelriemoliveira https://github.com/miguelriemoliveira, I'll try it out.

I'm still on a meeting. I need more 15 minutes. Can you meet at 15h?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/lardemua/AtlasCarCalibration/issues/146#issuecomment-623471473, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACWTHVTYZUDTUDVP6QJ4MT3RP3A3PANCNFSM4MXG63NQ .

[miguelriemoliveira]

I think it is ok if @eupedrosa is not there. I can show you the first steps of the objective function.

[aaguiar96]

Closing this issue since we are using the dataset obtained by @miguelriemoliveira, with two collections. Then, we are going to use a more accurate chessboard.