search

autonomousvision / carla_garage

[ICCV'23] Hidden Biases of End-to-End Driving Models
MIT License
203 stars 16 forks source link

Questions about reproducing Interfuser and TCP. #3

Closed SmashChen closed 1 year ago

SmashChen commented 1 year ago

Thank you for your great job! I noticed in the appendix that you tried to reproduce the results of Interfuser and TCP on the Carla leaderboard. How did you reproduce it? Do you use their pre-trained weights directly or use their code to retrain on your dataset.

Kait0 commented 1 year ago

For Interfuser we used their pre-trained weights, because no dataset was released. For TCP we downloaded their dataset and retrained the TCP model on all of their data.

SmashChen commented 1 year ago

What confuses me is why you used the pre-trained weights from Interfuser, but only got half the score?

Kait0 commented 1 year ago

Well, I am confused as well. We talked to the authors, they said the released model is not the same as the one they reported in their paper. They were vague on what the difference exactly is but emphasized that the released model was trained with less data. They did not want to release the model from the paper or help us reproduce their results, so we put the numbers from the released model in the paper.

SmashChen commented 1 year ago

Well，i get it. Thanks for your reply!