Inconsistency result of single chip radar pointcloud result

[JJQ7]

Description

I've encountered an issue where the point cloud results generated by [program name] are inconsistent with the results provided in the dataset [aspen11]. I'm trying to understand the cause of this discrepancy and how it might be resolved.

Steps to Reproduce

1. Downloaded the dataset [aspen11]
2. Ran the program [batch pointcloud generation]
3. Observed the resulting output

Expected Result

The point cloud data should closely match the provided pcl files in the dataset.

Actual Result

The generated point cloud data differs significantly from what is provided, with discrepancies in [specific details].

Environment

• OS: ubuntu

Could you please help clarify why there might be differences between the provided data and the program output? Any guidance on resolving this would be greatly appreciated.

Thank you for your assistance.

[azinke]

Hi @JJQ7 ! Please, how was the "Expected Result" point cloud generated? Could you also indicate what recording frame of aspen11 the pictures above are from?!

To my knowledge, no pointcloud is provided in the coloradar dataset for the cascaded chip radar. However, it contains heatmaps for the cascaded radar and pointcloud for the single chip radar recordings.

[JJQ7]

Thank you for your prompt reply!

1. how was the "Expected Result" point cloud generated?

This pointcloud image directly comes from the single chip pointcloud file in the coloradar dataset

2. Could you also indicate what recording frame of aspen11 the pictures above are from?

The picture is obtained by projecting the point cloud of all frames with groundtruth and observing the xy plane from a top-down perspective.

To my knowledge, no pointcloud is provided in the coloradar dataset for the cascaded chip radar. However, it contains heatmaps for the cascaded radar and pointcloud for the single chip radar recordings.

Yes. I agree with you. The comparison is conducted between the pointcloud provided in the dataset files and the pointcloud generated from the raw ADC files

[JJQ7]

In addition, the following changes are required to generate the pointcloud of single chip radar using Coloradar codes. core/record.py, line 173

# File: core/record.py, Line: 173

elif self._kwargs.get("save_as") == "bin":
    if self._sensor == "scradar":
        pcl = self.scradar.getPointcloudFromRaw(
            polar=self._kwargs.get("polar"))
        pcl.astype(np.float32).tofile(
            f"{self._output_dir}/radar_pointcloud_{idx}.bin")
        return idx                
    elif self._sensor == "ccradar":
        pcl = self.ccradar.getPointcloudFromRaw(
            polar=self._kwargs.get("polar"))
        pcl.astype(np.float32).tofile(
            f"{self._output_dir}/radar_pcl{idx}.bin")
        return idx

[azinke]

Thank you, I understand. That's a very nice idea.

Consider the two images below

(obtained with python coloradar.py --dataset aspen11 -i 300 --scradar --raw -pcl) Figure A

this is, for instance, a point cloud without the --bird-eye-view option. it's still unfiltered and merging the point cloud of all frames would not be clean.

The same frame from aspen11 with --bird-eye-view is show in the picture below (Figure B)

(obtained with python coloradar.py --dataset aspen11 -i 300 --scradar --raw -pcl -bev)

Figure B

Possible improvements

1. An improvement that could be made is to adjust the azimuth bins for the single-chip radar (in core/config.py) The default Azimuth and Elevation bins are a little high for the single-chip radar considering its resolution. The values were tuned to the cascaded radar (which has a higher resolution) and that's why the detections in Figure B look a little spread. I think you can decrease the values for the single-chip radar without problem

# core/config.py

# Minimum number of azimuth bins
NUMBER_AZIMUTH_BINS_MIN: int = 16

# Minimum number of elevation bins
NUMBER_ELEVATION_BINS_MIN: int = 32

But I guess by changing the azimuth bin, you'll also need to adjust the OS-CFAR parameters as well

# 1D OS-CFAR Parameters used for peak selection in Azimuth-FFT
AZ_OS_CFAR_WS: int = 16         # Window size
AZ_OS_CFAR_GS: int = 8          # Guard cell
AZ_OS_CFAR_TOS: int = 6         # Tos factor

Possible cause of the inconsistent pointclouds

1. Probably missing a reference (or base) coordinate frame

The point could for the radars generated from RAW data are in the coordinate frame of the radar itself. So to merge all frames using the pose information from the IMU, both data need to be in the same coordinate frame (like converted into a reference/base coordinate frame for instance). The transformation data provided by the dataset could be used for that.

Are you already converting the radar data (from raw) and the IMU data into the reference/base coordinate?!

[JJQ7]

Thank you for the detailed explanation and the suggestions for improving the point cloud visualization using the single-chip radar. Your insights are invaluable, and I appreciate the visual examples provided, which help clarify the current output and the potential enhancements.

Regarding the Adjustments to FFT parameters and OS-CFAR Parameters:

I agree that these parameters for the single-chip radar should be different from the cascade radar. I will implement these changes and observe how they affect the overall clarity of the point clouds.

Question about Coordinate Frame Conversion:

Your point about the possible cause of inconsistency due to the lack of a unified coordinate frame is very intriguing. To answer your question, I convert the Lidar pointcloud into the world frame by the same code. As shown in the following picture, the results confirm that the coordinate system should be correct.

Thank you once again for your guidance and support.

[azinke]

Hi @JJQ7! Good question. For mapping, I thought a base coordinate frame would be needed but maybe I'm wrong. However, I just had a quit look to the dataset paper https://arxiv.org/pdf/2103.04510 and in the description of the RADAR bin files there is the following:

Each of these files stores an array of 32 bit floating point values. Each radar point consists of 5 contiguous values: its (x, y, z) location in meters in the sensor’s local frame, its intensity value, and its range rate in meters per second

So, the values in the pointclouds provided by the dataset are also in the sensor's coordinate frame.

What might be different is the orientation of the axis between the pointcloud provided in the dataset and the ones generated

In Dataset

In the code, I've documented the following for the pointcloud generation method:

# core/radar.py, line 247
[0]: Azimuth
[1]: Range
[2]: Elevation
[3]: Velocity
[4]: Intensity of reflection in dB

And this in the drawn coordinate frame in the picture above, would be equivalent to (y, x, z) I'll check this later today as well, but you can already have a look and see if it's a possible cause

[JJQ7]

Good suggestion! @azinke

You’re right about the potential discrepancy in the orientation of the axes between the dataset-provided pointcloud and those generated by our code. To address this, I checked the projections quickly and found that a simple axis swap and inversion corrected the orientation:

# radar_pc_local is generated by the coloradar python code
x = radar_pc_local[i,1]
y = radar_pc_local[i,0]
radar_pc_local[i,0] = x
radar_pc_local[i,1] = -y

This adjustment realigned the generated pointclouds with those provided in the dataset. Here’s a snapshot of the modified point cloud projection, showing the corrected coordinate system:

Although there are noise points in the radar point cloud, the basic structure is similar to the lidar point cloud.

You are right about the coordinate system problem.

[azinke]

Awesome! Tuning the CFAR filter and the processing bins might help with the noises