Question regading seeming inconsistency of radar timestamps

[seamie6]

For each key frame, the timestamp of each radar seems to be random almost. As in the ordering of when they were captured along with the difference in time between successive captures seems random This is not seen for LiDAR and camera where the ordering of successive captures along with difference in timestamps are consistent for all key frames. I assume this has to do with the synchronization of LiDAR and camera, both operate at 20Hz such that cameras capture when the LiDAR sweeps across the center of its FOV. I understand that radar operates at 12Hz and so it is not synchronized with camera and LiDAR, and so the closest radar sweep is attributed to the key frame? But in that case I would assume there would exist some ordering across the sweeps between frames. Assuming there is a consistent ordering of sensors similar to the cameras, in that case for example: RADAR_FRONT_LEFT would always appear after RADAR_BACK_LEFT. But it does not, it seems random.

Here are some plots I made to visualise it easier for 3 successive key frames, only ms is plotted and they are normalised so the earliest capture is at 0ms:

Here is my code to generate it:

from nuscenes.nuscenes import NuScenes
from nuscenes.utils.geometry_utils import transform_matrix
import pandas as pd
import numpy as np

nusc = NuScenes(version='v1.0-mini', dataroot='/home/seamie/nuScenes', verbose=True)

# Specify which scene, sample, and instance you want to work with
scene = nusc.scene[6]
sample_token = scene['first_sample_token'] # get token for this scene
my_sample = nusc.get('sample', sample_token) # get first sample

# initialise dictionary
my_dict = {}

# iterate over all key frame
for _ in range(0,100):
    # iterate over all sensors
    for sensor_name, sensor_token in my_sample['data'].items():

        if sensor_name not in my_dict:
            # If the key doesn't exist, create a new list with the value
            my_dict[sensor_name] = []

        sensor_data = nusc.get('sample_data', sensor_token)

        time = pd.to_datetime(sensor_data['timestamp']*1000)
        # Extract minute, second, and millisecond
        minute = time.minute
        second = time.second
        millisecond = time.microsecond // 1000  # Convert microseconds to milliseconds

        # Store minute, second, and millisecond in a numpy array of shape (3, 1)
        time_array = millisecond

        # Append the numpy array to the dictionary
        my_dict[sensor_name].append(time_array)

    if my_sample['next']=='':
        break
    else:
        my_sample = nusc.get('sample', my_sample['next']) # get next sample

# plot sensor ms
import matplotlib.pyplot as plt
plt.rcParams['figure.dpi'] = 200

# Choose the timestep to plot (e.g., first timestep)
timestep_index = 3

# Extract milliseconds data for the chosen timestep for all sensors
sensor_names = list(my_dict.keys())

cam_lidar = [5,6,7,8,9,10,11]  # Example selected indexes
radar = [0,1,2,3,4]
all_ = [0,1,2,3,4,5,6,7,8,9,10,11]
sensor_names = [sensor_names[index] for index in radar]

ms_data = [my_dict[sensor_name][timestep_index] for sensor_name in sensor_names]

# Sort sensors and milliseconds data based on milliseconds data
sensor_names_sorted, ms_data_sorted = zip(*sorted(zip(sensor_names, ms_data), key=lambda x: x[1]))

# Find the minimum value
min_value = min(ms_data)

# Normalize the times by subtracting the minimum value
normalized_ms_data = [value - min_value for value in ms_data_sorted]

# Plot the bar chart
fig, ax = plt.subplots(figsize=(8, 6))
bars = ax.bar(sensor_names_sorted, normalized_ms_data)

# Set labels and title
ax.set_xlabel('Sensor')
ax.set_ylabel('Milliseconds')
ax.set_title(f'MS timestamp for key frame {timestep_index}')

# Rotate x-axis labels for better readability
plt.xticks(rotation=45, ha='right')

# Display exact number at the top of each bar
for bar in bars:
    height = bar.get_height()
    ax.text(bar.get_x() + bar.get_width() / 2, height, f'{height:.0f}', ha='center', va='bottom')

# Show plot
plt.tight_layout()
plt.show()

[whyekit-motional]

@seamie6 yes, I think you are correct - the radars in nuScenes are fired off in a random sequence, rather than a certain fixed sequence

[seamie6]

Thank you for the reply. Is there a specific reason they are fired off randomly? Rather than a fixed firing sequence.

[whyekit-motional]

@seamie6 I don't think there was any specific reason for it back then