Tracking problem

[JayChan-USTC]

For the tracking problem, why does the network not directly predict the 'tracking offset', but 'a two-dimensional velocity estimation'?

[tianweiy]

these two are the same stuff right? velocity * unit_time = offset. We exclude the z dim velocity here as it is close to zero ?

[JayChan-USTC]

The speed is predicted by the offset, and to a certain extent, there will be errors. I want to change the speed variable to offset. How do I reflect it in the centerpoint code? I don’t know how to use the point cloud information of the previous or next frames to add tracking offset tags to each training true value. Can you help me?

[tianweiy]

what error do we get? The time difference is the exact value and offset is just the difference * speed.

[tianweiy]

you can take a look at the velocity computation at that line https://github.com/tianweiy/CenterPoint/blob/e9ef04c3715aa3342fa42f4f4e064db987def6ad/det3d/datasets/nuscenes/nusc_common.py#L260 (need to find it in nuscenes-devkit) and do the change

[JayChan-USTC]

ok,thanks a lot!

[JayChan-USTC]

Excuse me, now there are two questions that confuse me. I don’t know whther my understanding is correct or not, I hope you can help me.

1. nuscences.py line379, 388,393 has the following code: ''current = self.get('sample_annotation', sample_annotation_token)'' ''first = self.get('sample_annotation', current['prev'])'' ''last = self.get('sample_annotation', current['next'])'' My understanding is that current, first and last are the data marked in the samples（sample_annotation is data annotation in sample), which are not from sweeps

2 In the centerPoint network, i do not know what the function of the point cloud data from sweeps=10. My understanding is that the function of point cloud datas from sweeps=10 is only to aggregate the labeled data in the sample because thre point is sparse, so that labeled data have enough point. And the point cloud data from sweeps=10 is not labeled.

Excuse me, are my two understandings correct?If there is something wrong, please give me additional explanation，Thanks!

[tianweiy]

1. My understanding is that current, first and last are the data marked in the samples（sample_annotation is data annotation in sample), which are not from sweeps

I think here it is sweep for prev / next. (each sample is a sweep)

2. as above, each sample is a sweep so it is aggregating one labeled frame + 9 unlabelled frame.

[JayChan-USTC]

1.In centerTrack, the input of the network is the image It at time t, the image I(t-1) at time t-1 and the heatmap at time t-1, and then the network predicts the tracking offset between the instance of the current frame and the instance of the previous frame . However, in CenterPoint, the input of the network is only one labeled frame + 9 unlabelled frame, and there is no previous frame information, how does the network accurately predict the precise speed of the current labeled frame？

2. https://github.com/tianweiy/CenterPoint/blob/e9ef04c3715aa3342fa42f4f4e064db987def6ad/tools/nusc_tracking/pub_tracker.py#:~:text=det%5B%27tracking%27%5D%20%3D%20np.array(det%5B%27velocity%27%5D%5B%3A2%5D)%20*%20-1%20*%20time_lag In the CenterPoint tracking problem , the tracking offset is calculated from the predicted speed, det['tracking'] = np.array(det['velocity'][:2]) -1 time_lag det['tracking'] is the tracking offset between the current frame and the previous frame(prev),And the prev is in sweep or in sample?（you mean each sample is a sweep）

3.you mean each sample is a sweep,but sample are annotated，sweeps are not marked by true values,right？

[tianweiy]

1. We use ten sweep lidar input for nuScenes, so you can also estimate local motion from the temporal point cloud (we get point measurements at different timestamp)

2. prev is sample

3. yes, sample has annotation, sweep doesn't.

4. yes,