Evaluation on the training model of custom dataset

Hi ! I'm tried to training 3d detection model using custom dataset, and I got the model. I want to evaluate the model, and used Waymo dataset evaluation method, after loading the data and modifying the classname. When I run test.py, I get this error message:

2021-09-28 11:05:25,704 INFO **********************Start logging********************** 2021-09-28 11:05:25,705 INFO CUDA_VISIBLE_DEVICES=ALL 2021-09-28 11:05:25,705 INFO cfg_file /mnt/NAS/liuyb/3D-detection/OpenPCDet/tools/cfgs/horizon_models/second_multihead.yaml 2021-09-28 11:05:25,705 INFO batch_size 2 2021-09-28 11:05:25,705 INFO workers 5 2021-09-28 11:05:25,705 INFO extra_tag default 2021-09-28 11:05:25,705 INFO ckpt /mnt/NAS/liuyb/3D-detection/OpenPCDet/output/ckpt/checkpoint_epoch_20.pth 2021-09-28 11:05:25,705 INFO launcher none 2021-09-28 11:05:25,705 INFO tcp_port 18888 2021-09-28 11:05:25,705 INFO local_rank 0 2021-09-28 11:05:25,705 INFO set_cfgs None 2021-09-28 11:05:25,705 INFO max_waiting_mins 30 2021-09-28 11:05:25,705 INFO start_epoch 0 2021-09-28 11:05:25,705 INFO eval_tag default 2021-09-28 11:05:25,705 INFO eval_all False 2021-09-28 11:05:25,705 INFO ckpt_dir None 2021-09-28 11:05:25,705 INFO save_to_file True 2021-09-28 11:05:25,705 INFO cfg.ROOT_DIR: /mnt/NAS/liuyb/3D-detection/OpenPCDet 2021-09-28 11:05:25,705 INFO cfg.LOCAL_RANK: 0 2021-09-28 11:05:25,705 INFO cfg.CLASS_NAMES: ['Car', 'Pedestrian', 'Cyclist', 'Bus', 'Truck', 'Construction', 'Tricycle', 'Blur', 'other'] 2021-09-28 11:05:25,705 INFO cfg.DATA_CONFIG = edict() 2021-09-28 11:05:25,705 INFO cfg.DATA_CONFIG.DATASET: CustomDataset 2021-09-28 11:05:25,705 INFO cfg.DATA_CONFIG.DATA_PATH: /mnt/NAS/HorizonDataset 2021-09-28 11:05:25,705 INFO cfg.DATA_CONFIG.POINT_CLOUD_RANGE: [-120, -20, -3, 180, 20, 1] 2021-09-28 11:05:25,705 INFO cfg.DATA_CONFIG.DATA_SPLIT = edict() 2021-09-28 11:05:25,705 INFO cfg.DATA_CONFIG.DATA_SPLIT.train: train 2021-09-28 11:05:25,705 INFO cfg.DATA_CONFIG.DATA_SPLIT.test: val 2021-09-28 11:05:25,705 INFO cfg.DATA_CONFIG.INFO_PATH = edict() 2021-09-28 11:05:25,705 INFO cfg.DATA_CONFIG.INFO_PATH.train: ['custom_infos_train.pkl'] 2021-09-28 11:05:25,705 INFO cfg.DATA_CONFIG.INFO_PATH.test: ['custom_infos_val.pkl'] 2021-09-28 11:05:25,705 INFO cfg.DATA_CONFIG.BALANCED_RESAMPLING: True 2021-09-28 11:05:25,705 INFO cfg.DATA_CONFIG.GET_ITEM_LIST: ['points'] 2021-09-28 11:05:25,705 INFO cfg.DATA_CONFIG.FOV_POINTS_ONLY: True 2021-09-28 11:05:25,705 INFO cfg.DATA_CONFIG.DATA_AUGMENTOR = edict() 2021-09-28 11:05:25,705 INFO cfg.DATA_CONFIG.DATA_AUGMENTOR.DISABLE_AUG_LIST: ['placeholder'] 2021-09-28 11:05:25,705 INFO cfg.DATA_CONFIG.DATA_AUGMENTOR.AUG_CONFIG_LIST: [{'NAME': 'gt_sampling', 'DB_INFO_PATH': ['custom_dbinfos_train.pkl'], 'PREPARE': {'filter_by_min_points': ['Car:10', 'Pedestrian:8', 'Cyclist:10', 'Bus:25', 'Truck:20', 'Construction:25', 'Tricycle:15', 'Blur:15', 'other:15']}, 'SAMPLE_GROUPS': ['Car:20', 'Pedestrian:15', 'Cyclist:15', 'Bus:30', 'Truck:25', 'Construction:30', 'Tricycle:20', 'Blur:20', 'other:15'], 'NUM_POINT_FEATURES': 4, 'DATABASE_WITH_FAKELIDAR': False, 'REMOVE_EXTRA_WIDTH': [0.0, 0.0, 0.0], 'LIMIT_WHOLE_SCENE': True}, {'NAME': 'random_world_flip', 'ALONG_AXIS_LIST': ['x']}, {'NAME': 'random_world_rotation', 'WORLD_ROT_ANGLE': [-0.78539816, 0.78539816]}, {'NAME': 'random_world_scaling', 'WORLD_SCALE_RANGE': [0.95, 1.05]}] 2021-09-28 11:05:25,706 INFO cfg.DATA_CONFIG.POINT_FEATURE_ENCODING = edict() 2021-09-28 11:05:25,706 INFO cfg.DATA_CONFIG.POINT_FEATURE_ENCODING.encoding_type: absolute_coordinates_encoding 2021-09-28 11:05:25,706 INFO cfg.DATA_CONFIG.POINT_FEATURE_ENCODING.used_feature_list: ['x', 'y', 'z', 'intensity'] 2021-09-28 11:05:25,706 INFO cfg.DATA_CONFIG.POINT_FEATURE_ENCODING.src_feature_list: ['x', 'y', 'z', 'intensity'] 2021-09-28 11:05:25,706 INFO cfg.DATA_CONFIG.DATA_PROCESSOR: [{'NAME': 'mask_points_and_boxes_outside_range', 'REMOVE_OUTSIDE_BOXES': True}, {'NAME': 'shuffle_points', 'SHUFFLE_ENABLED': {'train': True, 'test': False}}, {'NAME': 'transform_points_to_voxels', 'VOXEL_SIZE': [0.05, 0.05, 0.1], 'MAX_POINTS_PER_VOXEL': 5, 'MAX_NUMBER_OF_VOXELS': {'train': 20000, 'test': 40000}}] 2021-09-28 11:05:25,706 INFO cfg.DATA_CONFIG._BASE_CONFIG_: /mnt/NAS/liuyb/3D-detection/OpenPCDet/tools/cfgs/dataset_configs/custom_data.yaml 2021-09-28 11:05:25,706 INFO cfg.MODEL = edict() 2021-09-28 11:05:25,706 INFO cfg.MODEL.NAME: SECONDNet 2021-09-28 11:05:25,706 INFO cfg.MODEL.VFE = edict() 2021-09-28 11:05:25,706 INFO cfg.MODEL.VFE.NAME: MeanVFE 2021-09-28 11:05:25,706 INFO cfg.MODEL.BACKBONE_3D = edict() 2021-09-28 11:05:25,706 INFO cfg.MODEL.BACKBONE_3D.NAME: VoxelBackBone8x 2021-09-28 11:05:25,706 INFO cfg.MODEL.MAP_TO_BEV = edict() 2021-09-28 11:05:25,706 INFO cfg.MODEL.MAP_TO_BEV.NAME: HeightCompression 2021-09-28 11:05:25,706 INFO cfg.MODEL.MAP_TO_BEV.NUM_BEV_FEATURES: 256 2021-09-28 11:05:25,706 INFO cfg.MODEL.BACKBONE_2D = edict() 2021-09-28 11:05:25,706 INFO cfg.MODEL.BACKBONE_2D.NAME: BaseBEVBackbone 2021-09-28 11:05:25,706 INFO cfg.MODEL.BACKBONE_2D.LAYER_NUMS: [5, 5] 2021-09-28 11:05:25,706 INFO cfg.MODEL.BACKBONE_2D.LAYER_STRIDES: [1, 2] 2021-09-28 11:05:25,706 INFO cfg.MODEL.BACKBONE_2D.NUM_FILTERS: [128, 256] 2021-09-28 11:05:25,706 INFO cfg.MODEL.BACKBONE_2D.UPSAMPLE_STRIDES: [1, 2] 2021-09-28 11:05:25,706 INFO cfg.MODEL.BACKBONE_2D.NUM_UPSAMPLE_FILTERS: [256, 256] 2021-09-28 11:05:25,706 INFO cfg.MODEL.DENSE_HEAD = edict() 2021-09-28 11:05:25,706 INFO cfg.MODEL.DENSE_HEAD.NAME: AnchorHeadMulti 2021-09-28 11:05:25,706 INFO cfg.MODEL.DENSE_HEAD.CLASS_AGNOSTIC: False 2021-09-28 11:05:25,706 INFO cfg.MODEL.DENSE_HEAD.USE_DIRECTION_CLASSIFIER: True 2021-09-28 11:05:25,706 INFO cfg.MODEL.DENSE_HEAD.DIR_OFFSET: 0.78539 2021-09-28 11:05:25,706 INFO cfg.MODEL.DENSE_HEAD.DIR_LIMIT_OFFSET: 0.0 2021-09-28 11:05:25,706 INFO cfg.MODEL.DENSE_HEAD.NUM_DIR_BINS: 2 2021-09-28 11:05:25,706 INFO cfg.MODEL.DENSE_HEAD.USE_MULTIHEAD: True 2021-09-28 11:05:25,706 INFO cfg.MODEL.DENSE_HEAD.SEPARATE_MULTIHEAD: True 2021-09-28 11:05:25,706 INFO cfg.MODEL.DENSE_HEAD.ANCHOR_GENERATOR_CONFIG: [{'class_name': 'Car', 'anchor_sizes': [[4.63, 1.97, 1.74]], 'anchor_rotations': [0, 1.57], 'anchor_bottom_heights': [-0.95], 'align_center': False, 'feature_map_stride': 8, 'matched_threshold': 0.3, 'unmatched_threshold': 0.45}, {'class_name': 'Truck', 'anchor_sizes': [[6.93, 2.51, 2.84]], 'anchor_rotations': [0, 1.57], 'anchor_bottom_heights': [-0.6], 'align_center': False, 'feature_map_stride': 8, 'matched_threshold': 0.4, 'unmatched_threshold': 0.4}, {'class_name': 'Construction', 'anchor_sizes': [[6.37, 2.85, 3.19]], 'anchor_rotations': [0, 1.57], 'anchor_bottom_heights': [-0.225], 'align_center': False, 'feature_map_stride': 8, 'matched_threshold': 0.75, 'unmatched_threshold': 0.7}, {'class_name': 'Bus', 'anchor_sizes': [[10.5, 2.94, 3.47]], 'anchor_rotations': [0, 1.57], 'anchor_bottom_heights': [-0.085], 'align_center': False, 'feature_map_stride': 8, 'matched_threshold': 0.55, 'unmatched_threshold': 0.4}, {'class_name': 'Blur', 'anchor_sizes': [[10.29, 2.9, 3.87]], 'anchor_rotations': [0, 1.57], 'anchor_bottom_heights': [0.115], 'align_center': False, 'feature_map_stride': 8, 'matched_threshold': 0.9, 'unmatched_threshold': 0.7}, {'class_name': 'Tricycle', 'anchor_sizes': [[2.11, 0.77, 1.47]], 'anchor_rotations': [0, 1.57], 'anchor_bottom_heights': [-1.085], 'align_center': False, 'feature_map_stride': 8, 'matched_threshold': 0.65, 'unmatched_threshold': 0.5}, {'class_name': 'Cyclist', 'anchor_sizes': [[1.7, 0.6, 1.28]], 'anchor_rotations': [0, 1.57], 'anchor_bottom_heights': [-1.18], 'align_center': False, 'feature_map_stride': 8, 'matched_threshold': 0.5, 'unmatched_threshold': 0.35}, {'class_name': 'Pedestrian', 'anchor_sizes': [[0.73, 0.67, 1.77]], 'anchor_rotations': [0, 1.57], 'anchor_bottom_heights': [-0.935], 'align_center': False, 'feature_map_stride': 8, 'matched_threshold': 0.4, 'unmatched_threshold': 0.3}, {'class_name': 'other', 'anchor_sizes': [[0.41, 0.41, 1.07]], 'anchor_rotations': [0, 1.57], 'anchor_bottom_heights': [-1.285], 'align_center': False, 'feature_map_stride': 8, 'matched_threshold': 0.6, 'unmatched_threshold': 0.4}] 2021-09-28 11:05:25,706 INFO cfg.MODEL.DENSE_HEAD.SHARED_CONV_NUM_FILTER: 64 2021-09-28 11:05:25,706 INFO cfg.MODEL.DENSE_HEAD.RPN_HEAD_CFGS: [{'HEAD_CLS_NAME': ['Car']}, {'HEAD_CLS_NAME': ['Truck', 'Construction']}, {'HEAD_CLS_NAME': ['Bus', 'Blur']}, {'HEAD_CLS_NAME': ['Cyclist', 'Tricycle']}, {'HEAD_CLS_NAME': ['Pedestrian', 'other']}] 2021-09-28 11:05:25,706 INFO cfg.MODEL.DENSE_HEAD.TARGET_ASSIGNER_CONFIG = edict() 2021-09-28 11:05:25,706 INFO cfg.MODEL.DENSE_HEAD.TARGET_ASSIGNER_CONFIG.NAME: AxisAlignedTargetAssigner 2021-09-28 11:05:25,707 INFO cfg.MODEL.DENSE_HEAD.TARGET_ASSIGNER_CONFIG.POS_FRACTION: -1.0 2021-09-28 11:05:25,707 INFO cfg.MODEL.DENSE_HEAD.TARGET_ASSIGNER_CONFIG.SAMPLE_SIZE: 512 2021-09-28 11:05:25,707 INFO cfg.MODEL.DENSE_HEAD.TARGET_ASSIGNER_CONFIG.NORM_BY_NUM_EXAMPLES: False 2021-09-28 11:05:25,707 INFO cfg.MODEL.DENSE_HEAD.TARGET_ASSIGNER_CONFIG.MATCH_HEIGHT: False 2021-09-28 11:05:25,707 INFO cfg.MODEL.DENSE_HEAD.TARGET_ASSIGNER_CONFIG.BOX_CODER: ResidualCoder 2021-09-28 11:05:25,707 INFO cfg.MODEL.DENSE_HEAD.LOSS_CONFIG = edict() 2021-09-28 11:05:25,707 INFO cfg.MODEL.DENSE_HEAD.LOSS_CONFIG.LOSS_WEIGHTS = edict() 2021-09-28 11:05:25,707 INFO cfg.MODEL.DENSE_HEAD.LOSS_CONFIG.LOSS_WEIGHTS.cls_weight: 1.0 2021-09-28 11:05:25,707 INFO cfg.MODEL.DENSE_HEAD.LOSS_CONFIG.LOSS_WEIGHTS.loc_weight: 2.0 2021-09-28 11:05:25,707 INFO cfg.MODEL.DENSE_HEAD.LOSS_CONFIG.LOSS_WEIGHTS.dir_weight: 0.2 2021-09-28 11:05:25,707 INFO cfg.MODEL.DENSE_HEAD.LOSS_CONFIG.LOSS_WEIGHTS.code_weights: [1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0] 2021-09-28 11:05:25,707 INFO cfg.MODEL.POST_PROCESSING = edict() 2021-09-28 11:05:25,707 INFO cfg.MODEL.POST_PROCESSING.RECALL_THRESH_LIST: [0.3, 0.5, 0.7] 2021-09-28 11:05:25,707 INFO cfg.MODEL.POST_PROCESSING.MULTI_CLASSES_NMS: True 2021-09-28 11:05:25,707 INFO cfg.MODEL.POST_PROCESSING.SCORE_THRESH: 0.55 2021-09-28 11:05:25,707 INFO cfg.MODEL.POST_PROCESSING.OUTPUT_RAW_SCORE: False 2021-09-28 11:05:25,707 INFO cfg.MODEL.POST_PROCESSING.EVAL_METRIC: waymo 2021-09-28 11:05:25,707 INFO cfg.MODEL.POST_PROCESSING.NMS_CONFIG = edict() 2021-09-28 11:05:25,707 INFO cfg.MODEL.POST_PROCESSING.NMS_CONFIG.MULTI_CLASSES_NMS: True 2021-09-28 11:05:25,707 INFO cfg.MODEL.POST_PROCESSING.NMS_CONFIG.NMS_TYPE: nms_gpu 2021-09-28 11:05:25,707 INFO cfg.MODEL.POST_PROCESSING.NMS_CONFIG.NMS_THRESH: 0.1 2021-09-28 11:05:25,707 INFO cfg.MODEL.POST_PROCESSING.NMS_CONFIG.NMS_PRE_MAXSIZE: 4096 2021-09-28 11:05:25,707 INFO cfg.MODEL.POST_PROCESSING.NMS_CONFIG.NMS_POST_MAXSIZE: 500 2021-09-28 11:05:25,707 INFO cfg.OPTIMIZATION = edict() 2021-09-28 11:05:25,707 INFO cfg.OPTIMIZATION.BATCH_SIZE_PER_GPU: 4 2021-09-28 11:05:25,707 INFO cfg.OPTIMIZATION.NUM_EPOCHS: 20 2021-09-28 11:05:25,707 INFO cfg.OPTIMIZATION.OPTIMIZER: adam_onecycle 2021-09-28 11:05:25,707 INFO cfg.OPTIMIZATION.LR: 0.003 2021-09-28 11:05:25,707 INFO cfg.OPTIMIZATION.WEIGHT_DECAY: 0.01 2021-09-28 11:05:25,707 INFO cfg.OPTIMIZATION.MOMENTUM: 0.9 2021-09-28 11:05:25,707 INFO cfg.OPTIMIZATION.MOMS: [0.95, 0.85] 2021-09-28 11:05:25,707 INFO cfg.OPTIMIZATION.PCT_START: 0.4 2021-09-28 11:05:25,707 INFO cfg.OPTIMIZATION.DIV_FACTOR: 10 2021-09-28 11:05:25,707 INFO cfg.OPTIMIZATION.DECAY_STEP_LIST: [35, 45] 2021-09-28 11:05:25,707 INFO cfg.OPTIMIZATION.LR_DECAY: 0.1 2021-09-28 11:05:25,707 INFO cfg.OPTIMIZATION.LR_CLIP: 1e-07 2021-09-28 11:05:25,707 INFO cfg.OPTIMIZATION.LR_WARMUP: False 2021-09-28 11:05:25,707 INFO cfg.OPTIMIZATION.WARMUP_EPOCH: 1 2021-09-28 11:05:25,707 INFO cfg.OPTIMIZATION.GRAD_NORM_CLIP: 10 2021-09-28 11:05:25,708 INFO cfg.TAG: second_multihead 2021-09-28 11:05:25,708 INFO cfg.EXP_GROUP_PATH: mnt/NAS/liuyb/3D-detection/OpenPCDet/tools/cfgs/horizon_models &&&&&self.mode: test ***self.split: val -----split_dir: /mnt/NAS/HorizonDataset/val.txt 2021-09-28 11:05:25,782 INFO Loading custom dataset 2021-09-28 11:05:25,855 INFO Total samples for custom dataset: 2500 ***************generate_anchors anchors is torch.Size([1, 100, 750, 1, 2, 7]) ***************generate_anchors anchors is torch.Size([1, 100, 750, 1, 2, 7]) ***************generate_anchors anchors is torch.Size([1, 100, 750, 1, 2, 7]) ***************generate_anchors anchors is torch.Size([1, 100, 750, 1, 2, 7]) ***************generate_anchors anchors is torch.Size([1, 100, 750, 1, 2, 7]) ***************generate_anchors anchors is torch.Size([1, 100, 750, 1, 2, 7]) ***************generate_anchors anchors is torch.Size([1, 100, 750, 1, 2, 7]) ***************generate_anchors anchors is torch.Size([1, 100, 750, 1, 2, 7]) ***************generate_anchors anchors is torch.Size([1, 100, 750, 1, 2, 7]) 2021-09-28 11:05:27,473 INFO ==> Loading parameters from checkpoint /mnt/NAS/liuyb/3D-detection/OpenPCDet/output/ckpt/checkpoint_epoch_20.pth to GPU 2021-09-28 11:05:27,529 INFO ==> Checkpoint trained from version: pcdet+0.3.0+d3cd330+pyd687ba5 2021-09-28 11:05:27,682 INFO ==> Done (loaded 198/198) 2021-09-28 11:05:27,692 INFO *************** EPOCH 20 EVALUATION ***************** eval: 100%|██████████| 1250/1250 [03:22<00:00, 6.17it/s, recall_0.3=(0, 44046) / 144701] 2021-09-28 11:08:50,365 INFO *************** Performance of EPOCH 20 ***************** 2021-09-28 11:08:50,365 INFO Generate label finished(sec_per_example: 0.0811 second). 2021-09-28 11:08:50,365 INFO recall_roi_0.3: 0.000000 2021-09-28 11:08:50,365 INFO recall_rcnn_0.3: 0.304393 2021-09-28 11:08:50,365 INFO recall_roi_0.5: 0.000000 2021-09-28 11:08:50,365 INFO recall_rcnn_0.5: 0.293806 2021-09-28 11:08:50,365 INFO recall_roi_0.7: 0.000000 2021-09-28 11:08:50,365 INFO recall_rcnn_0.7: 0.249238 2021-09-28 11:08:50,366 INFO Average predicted number of objects(2500 samples): 19.177 2021-09-28 11:08:50.598582: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library libcudart.so.10.1 /home/liu/anaconda3/envs/env/lib/python3.7/site-packages/tensorflow/python/data/ops/iterator_ops.py:546: DeprecationWarning: Using or importing the ABCs from 'collections' instead of from 'collections.abc' is deprecated since Python 3.3,and in 3.9 it will stop working class IteratorBase(collections.Iterator, trackable.Trackable, /home/liu/anaconda3/envs/env/lib/python3.7/site-packages/tensorflow/python/data/ops/dataset_ops.py:106: DeprecationWarning: Using or importing the ABCs from 'collections' instead of from 'collections.abc' is deprecated since Python 3.3,and in 3.9 it will stop working class DatasetV2(collections.Iterable, tracking_base.Trackable, /home/liu/anaconda3/envs/env/lib/python3.7/site-packages/tensorflow/python/autograph/utils/testing.py:21: DeprecationWarning: the imp module is deprecated in favour of importlib; see the module's documentation for alternative uses import imp Start the waymo evaluation... Number: (pd, 47943) VS. (gt, 144110) Level 1: 144110, Level2: 0) WARNING:tensorflow:From /home/liu/anaconda3/envs/env/lib/python3.7/contextlib.py:82: TensorFlowTestCase.test_session (from tensorflow.python.framework.test_util) is deprecated and will be removed in a future version. Instructions for updating: Useself.session()orself.cached_session()` instead. 2021-09-28 11:08:51.501575: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library libcuda.so.1 2021-09-28 11:08:51.501673: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:982] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero 2021-09-28 11:08:51.501885: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1716] Found device 0 with properties: pciBusID: 0000:01:00.0 name: GeForce RTX 2070 SUPER computeCapability: 7.5 coreClock: 1.77GHz coreCount: 40 deviceMemorySize: 7.79GiB deviceMemoryBandwidth: 417.29GiB/s 2021-09-28 11:08:51.501908: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library libcudart.so.10.1 2021-09-28 11:08:51.501942: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library libcublas.so.10 2021-09-28 11:08:51.502833: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library libcufft.so.10 2021-09-28 11:08:51.503002: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library libcurand.so.10 2021-09-28 11:08:51.503862: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library libcusolver.so.10 2021-09-28 11:08:51.504285: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library libcusparse.so.10 2021-09-28 11:08:51.506022: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library libcudnn.so.7 2021-09-28 11:08:51.506081: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:982] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero 2021-09-28 11:08:51.506285: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:982] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero 2021-09-28 11:08:51.506440: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1858] Adding visible gpu devices: 0 2021-09-28 11:08:51.506643: I tensorflow/core/platform/cpu_feature_guard.cc:142] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN)to use the following CPU instructions in performance-critical operations: AVX2 FMA To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags. 2021-09-28 11:08:51.527423: I tensorflow/core/platform/profile_utils/cpu_utils.cc:104] CPU Frequency: 2899885000 Hz 2021-09-28 11:08:51.527795: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x5579ab886290 initialized for platform Host (this does not guarantee that XLA will be used). Devices: 2021-09-28 11:08:51.527807: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): Host, Default Version 2021-09-28 11:08:51.527927: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:982] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero 2021-09-28 11:08:51.528111: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1716] Found device 0 with properties: pciBusID: 0000:01:00.0 name: GeForce RTX 2070 SUPER computeCapability: 7.5 coreClock: 1.77GHz coreCount: 40 deviceMemorySize: 7.79GiB deviceMemoryBandwidth: 417.29GiB/s 2021-09-28 11:08:51.528143: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library libcudart.so.10.1 2021-09-28 11:08:51.528151: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library libcublas.so.10 2021-09-28 11:08:51.528170: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library libcufft.so.10 2021-09-28 11:08:51.528181: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library libcurand.so.10 2021-09-28 11:08:51.528191: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library libcusolver.so.10 2021-09-28 11:08:51.528200: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library libcusparse.so.10 2021-09-28 11:08:51.528211: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library libcudnn.so.7 2021-09-28 11:08:51.528249: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:982] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero 2021-09-28 11:08:51.528427: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:982] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero 2021-09-28 11:08:51.528577: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1858] Adding visible gpu devices: 0 2021-09-28 11:08:51.528598: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library libcudart.so.10.1 2021-09-28 11:08:51.811669: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1257] Device interconnect StreamExecutor with strength 1 edge matrix: 2021-09-28 11:08:51.811692: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1263] 0 2021-09-28 11:08:51.811699: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1276] 0: N 2021-09-28 11:08:51.811822: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:982] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero 2021-09-28 11:08:51.812035: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:982] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero 2021-09-28 11:08:51.812216: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:982] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero 2021-09-28 11:08:51.812383: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1402] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 2197 MB memory) -> physical GPU (device: 0, name: GeForce RTX 2070 SUPER, pci bus id: 0000:01:00.0, compute capability: 7.5) 2021-09-28 11:08:51.813716: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x5579c0ce0bf0 initialized for platform CUDA (this does not guarantee that XLA will be used). Devices: 2021-09-28 11:08:51.813726: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): GeForce RTX 2070 SUPER, Compute Capability 7.5 2021-09-28 11:08:51.877318: I waymo_open_dataset/metrics/ops/detection_metrics_ops.cc:157] Computing detection metrics for 47943 predicted boxes. 2021-09-28 11:08:51.877342: I waymo_open_dataset/metrics/ops/detection_metrics_ops.cc:159] Parsing prediction [47943,7][47943] 2021-09-28 11:08:51.888975: I waymo_open_dataset/metrics/ops/detection_metrics_ops.cc:168] Parsing ground truth [144110,7][144110] WARNING: Logging before InitGoogleLogging() is written to STDERR F0928 11:08:51.931784 24979 metrics_utils.cc:136] Check failed: shard < num_shards (5 vs. 4) Check failure stack trace: F0928 11:08:51.931789 24976 metrics_utils.cc:136] Check failed: shard < num_shards (5 vs. 4) Check failure stack trace: F0928 11:08:51.931789 24976 metrics_utils.cc:136] Check failed: shard < num_shards (5 vs. 4) F0928 11:08:51.931789 24973 metrics_utils.cc:136] Check failed: shard < num_shards (5 vs. 4) Check failure stack trace: F0928 11:08:51.931789 24976 metrics_utils.cc:136] Check failed: shard < num_shards (5 vs. 4) F0928 11:08:51.931789 24973 metrics_utils.cc:136] Check failed: shard < num_shards (5 vs. 4) F0928 11:08:51.931803 24980 metrics_utils.cc:136] Check failed: shard < num_shards (5 vs. 4) Check failure stack trace:

Process finished with exit code 134 (interrupted by signal 6: SIGABRT) ` This error seems to be issued from the waymo open dataset SDK，and the SDK and the waymo open dataset content are deeply bound, how should I modify it to meet the needs of my custom dataset model evaluation? Is there an easier way to evaluate the model? My custom dataset only has point cloud data, no images, only the detection results of point cloud data.

so I run test.py

open-mmlab / OpenPCDet

Evaluation on the training model of custom dataset #642