search

tarashakhurana / 4d-occ-forecasting

CVPR 2023: Official code for `Point Cloud Forecasting as a Proxy for 4D Occupancy Forecasting'
https://www.cs.cmu.edu/~tkhurana/ff4d/index.html
MIT License
212 stars 22 forks source link

Label of lidar points #3

Closed Jay-Ye closed 1 year ago

Jay-Ye commented 1 year ago

Hi, congrats for this great work. I have a minor question here. I noticed that in nuScenes, lidar points in class [1--23] are set to 1 (occupied), and the rest are 0 (empty). https://github.com/tarashakhurana/4d-occ-forecasting/blob/74549e2066d3f77aa2122f46fc30502b4d4bfbf9/data/nusc.py#L133

While the class [24--31] are:

24: 'flat.driveable_surface' 25: 'flat.other' 26: 'flat.sidewalk' 27: 'flat.terrain' 28: 'static.manmade' 29: 'static.other' 30: 'static.vegetation' 31: 'vehicle.ego'

Shouldn't points indicating these classes also be set to 1 (occupied) since the rays end here?

tarashakhurana commented 1 year ago

Hi, thanks for your question!

  1. The function load_fg_labels stores 1 for points that belong to foreground objects (objects that can move but might or might not be moving in the scene), and 0 for background (classes 24-31 that you pointed out). These labels are used to compute the FG/BG metrics later on during evaluation.
  2. They are not categorizing points into occupied or not occupied; all points belong to occupied regions and this is the entire premise of the differentiable voxel rendering algorithm.
  3. Finally, we do not have to worry about calling points on the ego-vehicle as FG or BG, as these points are removed from the point cloud when the data is loaded.
Jay-Ye commented 1 year ago

Oh, I get it. This label is not used in the training or testing but just for evaluation. Thank you so much for your help.