Bug in Merging Map Data with Trajectory Data(no predicted_map)

[zhepanqi]

when i ran adaptor.py,it just couldn't generate the predicted_map,which is needed in the training of HiVT,and this really confused me. I wonder if you once encounter the problem like this. Really hope i can get some help from you.

[zhepanqi]

When running HiVT, I encountered an error indicating that predicted_map was missing. Therefore, I added two lines of code in adaptor.py to check if this component exists in the generated map, and the result showed that it does not.

[alfredgu001324]

Thanks for pointing this out!

The first thing I noticed is that the MapTR GT file should also be on the full val set, so the two progress bars should have the same counts.

Can I take a look at the command you run? I wonder if there are any file mismatches here. Just FYI, there are indeed some missing frame infos since trajdata cannot fully capture all the frames. This is why there is a try except block there. However the missing data should not be too much. After filtering out this missing info, and the frames that do not have future and history (which is the majority of filtering), the # of scenes in full_val should be 4519 -> so 4519 valid scenarios out of 6019 frames.

[zhepanqi]

thank you so much for helping me,this is the code i ran

python adaptor.py \ --version mini \ --split mini_train \ --map_model MapTR \ --dataroot ../nuscenes \ --index_file ../adaptor_files/traj_scene_frame_mini_train.pkl \ --map_file ../mapping_results.pickle \ --gt_map_file ../adaptor_files/gt_mini_val.pickle \ --save_path ../trj_data/maptrv2

[alfredgu001324]

It should be --gt_map_file ../adaptor_files/gt_mini_train.pickle. Also, from your progress bar it seems that your mapping_results.pickle contains the estimated map of full val set, which does not match what you specified here (mini train).

[zhepanqi]

I'm sorry, but there is no gt_mini_train.pkl file in your cloud drive, so we chose to use val instead.

[zhepanqi]

and if i run the command like this, the same problem occur python adaptor.py \ --version trainval \ --split train \ --map_model MapTR \ --dataroot ../nuscenes \ --index_file ../adaptor_files/traj_scene_frame_full_val.pkl \ --map_file ../mapping_results.pickle \ --gt_map_file ../adaptor_files/gt_full_val.pickle \ --save_path ../trj_data/maptrv2

[alfredgu001324]

Uh I see, sorry for the oversight. I probably thought mini train is not as important so did not put it up.

As for your second command, the split should be val I guess (instead of train)?

[zhepanqi]

I'm very sorry to bother you multiple times. Actually, we are running the correct code, but we made a mistake in the comments section. Here is the part I re-ran, and you can see that this time I have used the complete dataset, but the issue of missing predicted_map still occurs.

Moreover, in the training of HiVT, we are also unable to proceed due to this issue.

[alfredgu001324]

No problem at all! The command looks good to me. I just tried re-running on my side but it seems to be fine. To debug, can you maybe try running this on val only, and see whether val has this problem? or maybe on the mini_val set.

From the printing message, it seems that all of the scenarios are missing predicted map data. Maybe quickly verify how many valid predicted maps there are in the mapping_results.pkl?

[zhepanqi]

Thank you very much for your perfect work and your patient responses. This has been very helpful for us, and we have almost successfully replicated some of your results. However, I would like to point out that not all data in adaptor.py can generate predicted_map. I suggest adding a filtering feature before saving the processed data, otherwise there will be errors during the subsequent HiVT model training.

[JT-Sun]

Thank you very much for your perfect work and your patient responses. This has been very helpful for us, and we have almost successfully replicated some of your results. However, I would like to point out that not all data in adaptor.py can generate predicted_map. I suggest adding a filtering feature before saving the processed data, otherwise there will be errors during the subsequent HiVT model training.

Hello, I have the same problem with you. How did you solve it and successfully train HiVT?

[zhepanqi]

Thank you very much for your perfect work and your patient responses. This has been very helpful for us, and we have almost successfully replicated some of your results. However, I would like to point out that not all data in adaptor.py can generate predicted_map. I suggest adding a filtering feature before saving the processed data, otherwise there will be errors during the subsequent HiVT model training.

Hello, I have the same problem with you. How did you solve it and successfully train HiVT?

As I said before, adding a filter before saving the ultimate data to make sure predicted_map exist may help you.

[alfredgu001324]

For debugging, can you maybe retrieve a sample token that corresponds to a pissing 'predicted_map', and then go back to mapping_results.pkl to see if there is any predicted map information corresponding to that sample token? As a reference, if things go smoothly this will generate a total of 15191 scenario files for full train.

[alfredgu001324]

Please refer to https://github.com/alfredgu001324/MapUncertaintyPrediction/issues/4 and https://github.com/alfredgu001324/MapUncertaintyPrediction/issues/13