search

nv-tlabs / lift-splat-shoot

Lift, Splat, Shoot: Encoding Images from Arbitrary Camera Rigs by Implicitly Unprojecting to 3D (ECCV 2020)
Other
1.06k stars 221 forks source link

Evaluation result #9

Open KuanchihHuang opened 3 years ago

KuanchihHuang commented 3 years ago

Thanks for sharing the excellent work! I have a question about the evaluation of the pretrained result. I got the result different from the paper. Do you have an idea how to correct it? Thank you!

> ======                                                                                                                                                                                 
> Loading NuScenes tables for version v1.0-mini...                                                                                                                                       
> 23 category,                                                                                                                                                                           
> 8 attribute,                                                                                                                                                                           
> 4 visibility,                                                                                                                                                                          
> 911 instance,                                                                                                                                                                          
> 12 sensor,                                                                                                                                                                             
> 120 calibrated_sensor,                                                                                                                                                                 
> 31206 ego_pose,                                                                                                                                                                        
> 8 log,                                                                                                                                                                                 
> 10 scene,                                                                                                                                                                              
> 404 sample,                                                                                                                                                                            
> 31206 sample_data,                                                                                                                                                                     
> 18538 sample_annotation,                                                                                                                                                               
> 4 map,                                                                                                                                                                                 
> Done loading in 0.348 seconds.                                                                                                                                                         
> ======                                                                                                                                                                                 
> Reverse indexing ...                                                                                                                                                                   
> Done reverse indexing in 0.1 seconds.                                                                                                                                                  
> ======                                                                                                                                                                                 
> NuscData: 323 samples. Split: train.                                                                                                                                                   
>                    Augmentation Conf: {'resize_lim': (0.193, 0.225), 'final_dim': (128, 352), 'rot_lim': (-5.4, 5.4), 'H': 900, 'W': 1600, 'rand_flip': True, 'bot_pct_lim': (0.0, 0.22
> ), 'cams': ['CAM_FRONT_LEFT', 'CAM_FRONT', 'CAM_FRONT_RIGHT', 'CAM_BACK_LEFT', 'CAM_BACK', 'CAM_BACK_RIGHT'], 'Ncams': 5}                                                              
> NuscData: 81 samples. Split: val.                                                                                                                                                      
>                    Augmentation Conf: {'resize_lim': (0.193, 0.225), 'final_dim': (128, 352), 'rot_lim': (-5.4, 5.4), 'H': 900, 'W': 1600, 'rand_flip': True, 'bot_pct_lim': (0.0, 0.22
> ), 'cams': ['CAM_FRONT_LEFT', 'CAM_FRONT', 'CAM_FRONT_RIGHT', 'CAM_BACK_LEFT', 'CAM_BACK', 'CAM_BACK_RIGHT'], 'Ncams': 5}                                                              
> Loaded pretrained weights for efficientnet-b0                                                                                                                                          
> loading model525000.pt                                                                                                                                                                 
> running eval...                                                                                                                                                                        
> {'loss': 0.12198955280545318, 'iou': 0.2699367443187027}
DRosemei commented 3 years ago

Thanks for sharing the excellent work! I have a question about the evaluation of the pretrained result. I got the result different from the paper. Do you have an idea how to correct it? Thank you!

> ======                                                                                                                                                                                 
> Loading NuScenes tables for version v1.0-mini...                                                                                                                                       
> 23 category,                                                                                                                                                                           
> 8 attribute,                                                                                                                                                                           
> 4 visibility,                                                                                                                                                                          
> 911 instance,                                                                                                                                                                          
> 12 sensor,                                                                                                                                                                             
> 120 calibrated_sensor,                                                                                                                                                                 
> 31206 ego_pose,                                                                                                                                                                        
> 8 log,                                                                                                                                                                                 
> 10 scene,                                                                                                                                                                              
> 404 sample,                                                                                                                                                                            
> 31206 sample_data,                                                                                                                                                                     
> 18538 sample_annotation,                                                                                                                                                               
> 4 map,                                                                                                                                                                                 
> Done loading in 0.348 seconds.                                                                                                                                                         
> ======                                                                                                                                                                                 
> Reverse indexing ...                                                                                                                                                                   
> Done reverse indexing in 0.1 seconds.                                                                                                                                                  
> ======                                                                                                                                                                                 
> NuscData: 323 samples. Split: train.                                                                                                                                                   
>                    Augmentation Conf: {'resize_lim': (0.193, 0.225), 'final_dim': (128, 352), 'rot_lim': (-5.4, 5.4), 'H': 900, 'W': 1600, 'rand_flip': True, 'bot_pct_lim': (0.0, 0.22
> ), 'cams': ['CAM_FRONT_LEFT', 'CAM_FRONT', 'CAM_FRONT_RIGHT', 'CAM_BACK_LEFT', 'CAM_BACK', 'CAM_BACK_RIGHT'], 'Ncams': 5}                                                              
> NuscData: 81 samples. Split: val.                                                                                                                                                      
>                    Augmentation Conf: {'resize_lim': (0.193, 0.225), 'final_dim': (128, 352), 'rot_lim': (-5.4, 5.4), 'H': 900, 'W': 1600, 'rand_flip': True, 'bot_pct_lim': (0.0, 0.22
> ), 'cams': ['CAM_FRONT_LEFT', 'CAM_FRONT', 'CAM_FRONT_RIGHT', 'CAM_BACK_LEFT', 'CAM_BACK', 'CAM_BACK_RIGHT'], 'Ncams': 5}                                                              
> Loaded pretrained weights for efficientnet-b0                                                                                                                                          
> loading model525000.pt                                                                                                                                                                 
> running eval...                                                                                                                                                                        
> {'loss': 0.12198955280545318, 'iou': 0.2699367443187027}

Have you sloved this problem? I met this, too.

jonahthelion commented 3 years ago

Are you using efficientnet_pytorch==0.7.0?

DRosemei commented 3 years ago

Are you using efficientnet_pytorch==0.7.0?

efficientnet_pytorch==0.7.1

jonahthelion commented 3 years ago

Using 0.7.0 should fix your problem. Sorry about that - I added that requirement to the README only recently https://github.com/nv-tlabs/lift-splat-shoot#preparation