Empirical testing suggests that the lidar.py part range data output may not be accurate

[TCIII]

SBC: NVIDIA Xavier NX OS: JP 4.5.1 DC: v 4.3 (dev)

The following hardware was used for the empirical testing: Rpi 4B/RPLidar A1M8_R6 (firmware 1.29)/Sainsmart OM5647 camera Nano 4GB/RPLidar A1M8-R5 (firmware 1.25)/Sainsmart IMX219 camera Xavier NX/RPLidar A2M8v(firmware 1.28)/D435i depth camera

The center of the RPLidar scanner was placed 46.5 inches (118 cm) from a mocha colored wall and using DC v 4.3, around 60 records were recorded for each of the three scanners.

The A1M8_R5, which is running the oldest Slamtec firmware, was probably the most accurate with several recorded distances of over 115 cm and a consistent starting range between 79 - 83 cm and consistent ending range between 118 - 121cm. Also the recorded ranges were reasonably consistent from index number to index number. The R5 recorded ~161 ranges. The A1M8_R6 and the A2M8 were the least accurate without a single value at or over 100 cm though the range accuracy from index value to index value was reasonable. The R6 recorded ~113 ranges. The A2M8 produced the least amount of range data. There were ~23 ranges recorded.

catalog_0_A1_R5_465_1.catalog.txt.zip catalog_0_A1_R6_465_1.catalog.txt.zip catalog_0_A2_465_2.catalog.txt.zip

[TCIII]

Conclusions: The Adafruit RPLidar library works well with the latest version of the Slamtec RPLidar firmware (1.29) and the older firmware version (1.25). I would question the accuracy of the range sorting process in the lidar.py part, which is a composite of several different lidar software sources, when used with the different Slamtec RPLidar versions. The reason I say this is I have done the following: Today I did A2M8 testing with Ezward's lidar_test.py program that prints both the angle and the accompanying range. First I captured one full scan (EM_A2_M8_Test_01) of the A2M8 looking straight ahead at the front of our office. Then I captured one full scan (EM_A2_M8_Test_02) of the A2M8 still looking straight ahead as before, but with a 13 inch wide box positioned 46.5 inches (~118cm -119cm/118X.Xmm/119X.Xmm) from the center of the scanner turret. Boy was I surprised when I looked at the data from the scan with the target positioned directly in front of the scanner at 46.5 inches. I think that this data angle/range string proves that the A2M8 start of scan (0/360 deg) is at the front of the scanner on a centerline running from the cable end through the scanner axis and out the front of the scanner: [(0.15625, 1190.0), (1.40625, 1190.5), (2.03125, 1191.75), (3.578125, 1196.25), (350.65625, 1196.0), (351.5625, 1196.75), (352.8125, 1190.0), (353.703125, 1192.5), (354.96875, 1188.75), (355.90625, 1190.5), (356.78125, 1189.25), (357.109375, 1188.0), (358.015625, 1188.5), (359.34375, 1188.75)]

EM_A2_M8_Test_01.zip EM_A2_M8_Test_02.zip

[TCIII]

I then proceeded to go through the same exercise with a RPLidar A1M8-R6 (firmware 1.29). As before, when I looked at the data from the scan with the target positioned directly in front of the scanner at 46.5 inches (~118X.Xmm/119X.Xmm) I came to the same conclusion as with the A2M8. I think that this data angle/range string proves that the A1M8 start of scan (0/360 deg) is at the front of the scanner on a centerline running from the motor end through the scanner axis and out the front of the scanner: [(0.109375, 1186.0), (1.046875, 1186.25), (2.484375, 1188.0), (3.890625, 1190.5), (4.15625, 1190.0), (5.34375, 1192.75), (6.765625, 1197.75), (352.953125, 1194.75), (353.96875, 1196.75), (354.0625, 1199.75), (355.34375, 1190.0), (356.78125, 1188.5), (357.03125, 1188.0), (358.21875, 1186.75), (359.6875, 1185.25)]

EM_A1_M8_Test_01.zip EM_A1_M8_Test_02.zip

[TCIII]

The above empirical testing definitely validates the Adafruit RPLidar library functionality and the fact that both the A1M8 and the A2M8 Slamtec RPLidars function as intended by the Manufacturer. It is also further proof that there are definite issues with the way the lidar.py part processes the range data from the different versions of the Slamtec RPLidars. Additionally it proves that Slamtec starts the clockwise scan (0 degrees) of both the A1M8 and the A2M8 on the scanner centerline at the front of the scanner and ends the scan (360 degrees) on the centerline at the front of the scanner. In the case of the A1M8, the RPLidar is mounted with the scanner turret facing forward with the motor at the back of the Lidar chassis. The A2M8 is mounted with the scanner turret facing forward with the cable at the back of the Lidar mount.

[Ezward]

See my RPLidar experiments at https://github.com/Ezward/rplidar

I did some work to see if we could stream in measurements rather that wait for an entire 360 degree scan. See stream_lidar.py for an example that streams measurements as they are aquired. This will do filtering, but it does not sort.

I used this to look at the performance characteristics of the RPLidar A1M8-R6 using scan defaults.

Here is an example where I streamed 10 full scans, then I used grep to count the number of points in each scan. Scan 0 usually has about 250 or more points. Subsequent scans have 267 or 268 measurements. This is using the default scan settings and including zero readings.

$ python stream_lidar.py -n 10 -d 0 >stream.log
$ cat stream.log | grep scan | wc -l
$ cat stream.log | grep '"scan": 0,' | wc -l
255
$ cat stream.log | grep '"scan": 1,' | wc -l
268
$ cat stream.log | grep '"scan": 2,' | wc -l
268
$ cat stream.log | grep '"scan": 3,' | wc -l
267
$ cat stream.log | grep '"scan": 4,' | wc -l
268
$ cat stream.log | grep '"scan": 5,' | wc -l
267
$ cat stream.log | grep '"scan": 6,' | wc -l
268
$ cat stream.log | grep '"scan": 7,' | wc -l
267
$ cat stream.log | grep '"scan": 8,' | wc -l
268
$ cat stream.log | grep '"scan": 9,' | wc -l
267
$ cat stream.log | grep '"scan": 10,' | wc -l
267

I added time to each measurement as they came in. Here are the first and last points in scan 1:

(venv) pi@raspberrypi:~/projects/rplidar $ head -n 8 stream.1.log
{
  "scan": 1,
  "time": 1628576979.515709,
  "distance": 0.0,
  "angle": 359.21875,
  "x": 0.0,
  "y": -0.0
}
(venv) pi@raspberrypi:~/projects/rplidar $ tail -n 8 stream.1.log
{
  "scan": 1,
  "time": 1628576979.652241,
  "distance": 0.0,
  "angle": 357.78125,
  "x": 0.0,
  "y": -0.0
}

The difference there is 0.136532068 seconds or 136.5ms. So with scan defaults the rate for full scans is 1/0.136532068 = 7.3 scans per second.

Scan 1 has 266 measurements.

$ cat stream.log | grep '"scan": 1,' | wc -l
266

So that is 266/0.136532068 = 1948 measurements per second. That surprises me because the marketing material says the device can do 8000 measurements per second.

It seems that the points are almost sorted by angle as delivered, but not quite. Here is an example from scan 1 where the points are not delivered in order. I think this is why there is logic in the donkey part to sort them. However, I don't really see any reason why we need them sorted. However, this does defeat that logic I was going to use to detect the start of a scan where you would just check for a new measurement whose angle is less than the previous measurement. You can see in the code that I did use the 'new_scan' boolean provided by the lidar.

{
  "scan": 1,
  "time": 1628576979.6257262,
  "distance": 0.0,
  "angle": 284.46875,
  "x": 0.0,
  "y": -0.0
}
{
  "scan": 1,
  "time": 1628576979.6261306,
  "distance": 0.0,
  "angle": 292.96875,
  "x": 0.0,
  "y": -0.0
}
{
  "scan": 1,
  "time": 1628576979.6282256,
  "distance": 1686.5,
  "angle": 287.15625,
  "x": 497.48129249332277,
  "y": -1611.4572950032443
}

So there is probably a little work to see if we can up the scan rate to 10hz and increase the density of the data points.

[TCIII]

@Ezward,

Because of the ambiguity of where the 0/360 degree reference is on the Slamtec RPLidars, I believe it is in our best interests to reach a consensus on where the Slamtec RPLidars' 0/360 degree physical reference point actually is before moving forward. Comments?

[TCIII]

@Ezward, "So that is 266/0.136532068 = 1948 measurements per second. That surprises me because the marketing material says the device can do 8000 measurements per second." I suspect that the 8000 measurements/second rate is arrived at using the Slamtec SDK on a I7 or faster PC?

Also, maybe we should report any further scan analysis on DC issue #910 and prepare to close this issue out as "solved"?

[Ezward]

So I think you are saying is that zero degrees is marked on the diagrams using the x-axis arrow, in both cases pointing down. And also, since you would normally mount the lidar on a car with those facing at the back of the car, then the forward angle is actually 180 degrees. Thanks for figuring this out. So that means we need a configuration property for the angle offset so we can normalize angles to that zero is always forward. In addition, the lidar is spinning clockwise, so angles are increasing opposite of what we would expect using standard math.

This second thing; the direction of the lidar spin, seems to be an attribute of the lidar model. So we can bake that into the code for that lidar model. I would think we want angles to increase in a counter-clockwise direction so normal math rules apply. So we would adjust the angle angle = (360 - (angle % 360)) % 360

The first thing, the position of the zero angle relative to the forward direction of the car, depends upon both on the lidar model and how the user mounts that lidar on their car. So that needs to be put into configuration and we need to adjust angles before we output them so that zero is always forward on the car. Assuming we have angles that increase in a counter-clockwise direction; angle = (angle - FORWARD_ANGLE + 360) % 360

@TCIII does that sound like it is consistent with your findings?

[TCIII]

@Ezward,

I don't agree that the "forward angle is at 180 degrees" for the following reasons:

Based on the following angle/range data, captured by your lidar_test.py program, when using a target 13 inches wide that is centered directly in front of the A1M8 scanner and is 46.5 inches (118X.X - 119X.X mm) from the center of the scanner:

[(0.109375, 1186.0), (1.046875, 1186.25), (2.484375, 1188.0), (3.890625, 1190.5), (4.15625, 1190.0), (5.34375, 1192.75), (6.765625, 1197.75), (352.953125, 1194.75), (353.96875, 1196.75), (354.0625, 1199.75), (355.34375, 1190.0), (356.78125, 1188.5), (357.03125, 1188.0), (358.21875, 1186.75), (359.6875, 1185.25)]

I would say that the A1M8 start of scan (0/360 deg) is at the front of the scanner on a centerline running from the motor end through the scanner axis and out the front of the scanner.

I say this because the 0 degree through 6 degree ranges are around 118X to 119X mm and the 352 degree through the 359 degree ranges are 118X -119X mm. The first thing the scanner sees at 0 degrees, during the clockwise rotation, is the target at 118X mm and when beyond 6 degrees falls off the target at 119X mm. Then, after rotating clockwise towards the front of the scanner chassis, the scanner sees the target again at 352 through 359 degrees where the range varies from 119X through 118X mm as the scanner moves from the left edge of the target towards the center of target at 0 degrees to start the scan over again in the clockwise direction.

If the forward angle is at 180 degrees, how do you explain the 0 degree range of 1186.0 mm which is almost the same distance (118X mm) that the target is from the scanner at the front of the scanner?

Here are the ranges that the A1M8 scanner saw starting at 180 degrees:

(180.9375, 107.75), (181.703125, 107.75), (182.375, 108.0), (183.890625, 108.0), (184.515625, 108.0), (185.140625, 108.0), (186.875, 108.25), (187.265625, 108.5), 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, (198.40625, 4886.5).

Directly behind the scanner is a two inch wide post that supports the Sainsmart WFOV camera above the scanner, which probably accounts for the 0 mm range at 188 degrees because the support post is about 10.8 cm from the center of the scanner and well within the minimum range of 15 cm.

Additionally, since the scanner rotates in a clockwise direction, the angle increase from 0 degrees to 360 degrees as the scanner rotates though it range of travel.

Your reference would work if the back of the scanner on the centerline was assumed to be 0 degrees with the scanner windows looking forward at 180 degrees. I think it all boils down to where the 0/360 degree mark is assumed to be located. My degree/range data shows the target at 0 degrees which is at the front of the scanner FOV.

Comments?

[Ezward]

I don't understand what is front or back of scanner, but I think you are saying the zero heading is pointing up in the diagrams (opposite of what I was saying). That makes sense. I think everything I suggested related to configurations properties still holds. In this case, if the A1M8 is pointed so the motor faces the back of the car, then the zero heading is facing directly front and so FORWARD_ANGLE would be zero. If the A2 has it's interface lead pointing towards the back of the car, then again the FORWARD_ANGLE would be zero. That all makes sense since that is the most intuitive way to mount the lidars.

[TCIII]

@Ezward,

That is a good summation. I agree with the Lidar physical mounting references.

I look at it this way:

The A1M8 angle/range data recorded the target at 0 degrees when the laser in the scanner was looking forward at the target at a recorded 0 degrees. Now comes the tricky part. Is 0 degrees at the motor end of the chassis or is it at the front end of the chassis with the laser in the scanner looking forward?

Chris and the UCSD 2019 Team students assumed that 0 degrees started at the motor end and the number of degrees increased towards the 180 degree point which would be looking out the front of the vehicle, however where was the laser looking at 0 and 180 degrees? They offered no evidence of where it was looking.

We know for a fact, through empirical testing, that the laser is looking forward when the target range is recorded at 0 degrees and is looking backwards towards the motor end when the target range is recorded at 180 degrees.

Therefore I believe that we can pick whichever end of the chassis we want for the 0 degrees point as long as we stick with the convention we ultimately choose. If we chose the motor end then when angle and range is recorded at 0 degrees the laser is looking forward and when angle and range is recorded at 180 degrees the laser is looking backwards. The logic is still the same if we chose the front end for the 0 degrees point which makes the motor end the 180 degrees point. I personally like the front end to be the 0 degrees point as it jibes with the Slamtec A1M8 diagram irrespective of where the "X" leg is pointing as this is probably a ROS convention.

Comments?

[Ezward]

I agree we should recommend that user's mount their lidar in this 'natural' way where zero heading is pointing forward, but that is not mandatory. We can provide the FORWARD_ANGLE configuration property so they can adjust if they mounted it in a different way.

[Ezward]

See my RPLidar experiments at https://github.com/Ezward/rplidar

I did some work to see if we could stream in measurements rather that wait for an entire 360 degree scan. See stream_lidar.py for an example that streams measurements as they are aquired. This will do filtering, but it does not sort.

I used this to look at the performance characteristics of the RPLidar A1M8-R6 using scan defaults.

Here is an example where I streamed 10 full scans, then I used grep to count the number of points in each scan. Scan 0 usually has about 250 or more points. Subsequent scans have 267 or 268 measurements. This is using the default scan settings and including zero readings.

$ python stream_lidar.py -n 10 -d 0 >stream.log
$ cat stream.log | grep scan | wc -l
$ cat stream.log | grep '"scan": 0,' | wc -l
255
$ cat stream.log | grep '"scan": 1,' | wc -l
268
$ cat stream.log | grep '"scan": 2,' | wc -l
268
$ cat stream.log | grep '"scan": 3,' | wc -l
267
$ cat stream.log | grep '"scan": 4,' | wc -l
268
$ cat stream.log | grep '"scan": 5,' | wc -l
267
$ cat stream.log | grep '"scan": 6,' | wc -l
268
$ cat stream.log | grep '"scan": 7,' | wc -l
267
$ cat stream.log | grep '"scan": 8,' | wc -l
268
$ cat stream.log | grep '"scan": 9,' | wc -l
267
$ cat stream.log | grep '"scan": 10,' | wc -l
267

I added time to each measurement as they came in. Here are the first and last points in scan 1:

(venv) pi@raspberrypi:~/projects/rplidar $ head -n 8 stream.1.log
{
  "scan": 1,
  "time": 1628576979.515709,
  "distance": 0.0,
  "angle": 359.21875,
  "x": 0.0,
  "y": -0.0
}
(venv) pi@raspberrypi:~/projects/rplidar $ tail -n 8 stream.1.log
{
  "scan": 1,
  "time": 1628576979.652241,
  "distance": 0.0,
  "angle": 357.78125,
  "x": 0.0,
  "y": -0.0
}

The difference there is 0.136532068 seconds or 136.5ms. So with scan defaults the rate for full scans is 1/0.136532068 = 7.3 scans per second.

Scan 1 has 266 measurements.

$ cat stream.log | grep '"scan": 1,' | wc -l
266

So that is 266/0.136532068 = 1948 measurements per second. That surprises me because the marketing material says the device can do 8000 measurements per second.

It seems that the points are almost sorted by angle as delivered, but not quite. Here is an example from scan 1 where the points are not delivered in order. I think this is why there is logic in the donkey part to sort them. However, I don't really see any reason why we need them sorted. However, this does defeat that logic I was going to use to detect the start of a scan where you would just check for a new measurement whose angle is less than the previous measurement. You can see in the code that I did use the 'new_scan' boolean provided by the lidar.

{
  "scan": 1,
  "time": 1628576979.6257262,
  "distance": 0.0,
  "angle": 284.46875,
  "x": 0.0,
  "y": -0.0
}
{
  "scan": 1,
  "time": 1628576979.6261306,
  "distance": 0.0,
  "angle": 292.96875,
  "x": 0.0,
  "y": -0.0
}
{
  "scan": 1,
  "time": 1628576979.6282256,
  "distance": 1686.5,
  "angle": 287.15625,
  "x": 497.48129249332277,
  "y": -1611.4572950032443
}

So there is probably a little work to see if we can up the scan rate to 10hz and increase the density of the data points.

[Ezward]

I did find another issue regarding accuracy of the current RPLidar implementation the LidarPlot part that plots the lidar. The LidarPlot assumes that angles increase counter-clockwise as is normal in mathematics. However, the RPLidar spins in a clockwise direction and so angles appear to increase clockwise. So the calculation of the (x, y) Cartesian coordinate pair in LidarPlot is not correct for RPLidar. I'm currently creating a new part that addresses that issue, but that is also a source of error is anyone was depending upon that plot (or otherwise assuming that the angles were valid in most mathematical models).

[TCIII]

@Ezward,

Excellent work to say the least. I tried your stream_lidary.py program and it worked flawlessly and returned results similar to yours.

[Ezward]

@TCIII PR https://github.com/autorope/donkeycar/pull/918 merged my new lidar stuff. Can we close this issue?

[TCIII]

@Ezward, Yes.