evaluate nuscenes

[NileLEE1]

when i want to evaluate nuscenes, but it is wrong. it seems to do the argoverse evaluate. can u help me. thank you so much

[NileLEE1]

GPU_NUM=1; \ MODEL_PATH=checkpoints/models.laformer.1/model_save/model.50.bin; \ python src/eval.py --future_frame_num 12 --eval_batch_size 128 \ --output_dir ${OUTPUT_DIR} --hidden_size 64 --train_batch_size 32 \ --lane_loss_weight 0.9 --topk 2 --reuse_temp_file --model_recover_path 50 \ --distributed_training ${GPU_NUM} --other_params step_lane_score stage_two \ semantic_lane direction enhance_global_graph subdivide new laneGCN \ point_level-4-3 stage-two-train_recover=${MODEL_PATH} stage-two-epoch=50 \ nuscenes nuscenes_mode_num=5 --do_eval {'add_prefix': None, 'core_num': 1, 'cuda_visible_device_num': None, 'data_dir': 'val/data/', 'data_dir_for_val': 'val/data/', 'debug': False, 'distributed_training': 1, 'do_eval': True, 'do_test': False, 'do_train': False, 'eval_batch_size': 128, 'eval_params': ['mode_num=5'], 'future_frame_num': 12, 'future_test_frame_num': 16, 'global_graph_depth': 1, 'hidden_dropout_prob': 0.1, 'hidden_size': 64, 'historical_steps': 20, 'initializer_range': 0.02, 'lane_loss_weight': 0.9, 'learning_rate': 0.001, 'log_dir': 'checkpoints/models.laformer.nuscenes.1', 'master_port': '12355', 'max_distance': 50.0, 'method_span': [0, 1], 'mode_num': 5, 'model_recover_path': 'checkpoints/models.laformer.nuscenes.1/model_save/model.50.bin', 'model_save_dir': 'checkpoints/models.laformer.nuscenes.1/model_save', 'no_cuda': False, 'num_train_epochs': 16.0, 'other_params': {'step_lane_score': True, 'stage_two': True, 'semantic_lane': True, 'direction': True, 'enhance_global_graph': True, 'subdivide': True, 'new': True, 'laneGCN': True, 'point_level-4-3': True, 'stage-two-train_recover': 'checkpoints/models.laformer.1/model_save/model.50.bin', 'stage-two-epoch': 50, 'nuscenes': True, 'nuscenes_mode_num': 5}, 'output_dir': 'checkpoints/models.laformer.nuscenes.1', 'reuse_temp_file': True, 'seed': 777, 'sub_graph_depth': 3, 'subdivide_length': 5, 'temp_file_dir': 'checkpoints/models.laformer.nuscenes.1/temp_file', 'topk': 2, 'train_all': False, 'train_batch_size': 32, 'train_extra': False, 'train_params': [], 'use_centerline': False, 'use_map': False, 'vector_size': 32, 'visualize': False, 'weight_decay': 0.01, 'z_size': 2}

11/13/2023 06:19:30 - INFO - main - args output_dir checkpoints/models.laformer.nuscenes.1 other_params ['step_lane_score', 'stage_two', 'semantic_lane', 'direction', 'enhance_global_graph', 'subdivide', 'new', 'laneGCN', 'point_level-4-3', ('stage-two-train_recover', 'checkpoints/models.laformer.1/model_save/model.50.bin'), ('stage-two-epoch', 50), 'nuscenes', ('nuscenes_mode_num', 5)] 11/13/2023 06:19:30 - INFO - main - device: cpu Loading Evalute Dataset ['val/data/'] Traceback (most recent call last): File "src/eval.py", line 138, in main() File "src/eval.py", line 134, in main do_eval(args) File "src/eval.py", line 77, in do_eval eval_dataset = Dataset(args, args.eval_batch_size) File "/fs/scratch/XCSERVER_AI-Initiative/aic2sgh/LAformer-main/src/datascripts/dataset_argoverse.py", line 428, in init pickle_file = open(os.path.join(args.temp_file_dir, get_name('ex_list')), 'rb') FileNotFoundError: [Errno 2] No such file or directory: 'checkpoints/models.laformer.nuscenes.1/temp_file/eval.ex_list'

[mengmengliu1998]

The reported error looks like you didn't preprocess the nuScenes evaluation set. Or you didn't give the correct path to the preprocessed file. Make sure there are preprocessed files under this path: checkpoints/models.laformer.nuscenes.1/temp_file/eval.ex_list

[NileLEE1]

OK. thank u

[NileLEE1]

i have another question if the model in checkpoint is your sota in eval.ai

[NileLEE1]

OUTPUT_DIR=checkpoints/models.laformer.nuscenes.1; GPU_NUM=1; MODEL_PATH=checkpoints/models.laformer.1/model_save/model.50.bin; python src/eval.py --future_frame_num 12 --eval_batch_size 128 --output_dir ${OUTPUT_DIR} --hidden_size 64 --train_batch_size 32 --lane_loss_weight 0.9 --topk 2 --reuse_temp_file --model_recover_path 50 --distributed_training ${GPU_NUM} --other_params step_lane_score stage_two semantic_lane direction enhance_global_graph subdivide new laneGCN point_level-4-3 stage-two-train_recover=${MODEL_PATH} stage-two-epoch=50 nuscenes nuscenes_mode_num=5 submission --do_eval i add submission but i dont find the result can u help me

[mengmengliu1998]

i have another question if the model in checkpoint is your sota in eval.ai Yes, you can refer to https://github.com/mengmengliu1998/LAformer/blob/main/src/eval.py#L115