JeffWang987 / OpenOccupancy

[ICCV 2023] OpenOccupancy: A Large Scale Benchmark for Surrounding Semantic Occupancy Perception
Apache License 2.0
551 stars 49 forks source link

low eval performence #10

Closed qihao-plus closed 1 year ago

qihao-plus commented 1 year ago

the eval performence is weird, reproduce based on commit

epoch 19 (set loss_norm=False)

2023-03-15 06:03:11,898 - mmdet - INFO - SC Evaluation                                                                                                                                        
2023-03-15 06:03:11,899 - mmdet - INFO - +-----------+-------+                                                                                                                                
|   class   |  IoU  |                                                                                                                                                                         
+-----------+-------+                                                                                                                                                                         
| non-empty | 0.163 |                                                                                                                                                                         
+-----------+-------+                                                                                                                                                                         
2023-03-15 06:03:11,899 - mmdet - INFO - SSC Evaluation                                                                                                                                       
2023-03-15 06:03:11,899 - mmdet - INFO - +----------------------+-------+                                                                                                                     
|        class         |  IoU  |                                                                                                                                                              
+----------------------+-------+                                                                                                                                                              
|         free         | 0.917 |                                                                                                                                                              
|       barrier        | 0.085 |                                                                                                                                                              
|       bicycle        | 0.044 |                                                                                                                                                              
|         bus          | 0.099 |                                                                                                                                                              
|         car          | 0.116 |                                                                                                                                                              
| construction_vehicle | 0.048 |                                                                                                                                                              
|      motorcycle      | 0.071 |                                                                                                                                                              
|      pedestrian      |  0.07 |                                                                                                                                                              
|     traffic_cone     | 0.041 |                                                                                                                                                              
|       trailer        | 0.042 |                                                                                                                                                              
|        truck         | 0.093 |                                                                                                                                                              
|  driveable_surface   | 0.215 |                                                                                                                                                              
|      other_flat      | 0.139 |                                                                                                                                                              
|       sidewalk       | 0.134 |                                                                                                                                                              
|       terrain        | 0.126 |                                                                                                                                                              
|       manmade        | 0.062 |                                                                                                                                                              
|      vegetation      | 0.092 |                                                                                                                                                              
|         mean         | 0.092 |                                                                                                                                                              
+----------------------+-------+  
qihao-plus commented 1 year ago

close it since I find the C-baseline is also very low in paper.