felipecode
commented
2 years ago There are several reasons. Listing some:
- You are using a very different CARLA version. CARLA changes considerably the pedestrians and car models and that can affect the results. The results were obtained on the old CARLA 0.8.X version.
- The models used are listed and explained here https://github.com/felipecode/coiltraine/blob/master/docs/exploring_limitations.md. Make sure to use the correct one for reproducing.
- Make sure you match also the frame rate and configs for the server configuration.
I am using the nocrash resnetimnetS1 model as a baseline and have run tests using checkpoint 660000. I am finding that my results for the New Town experiments are much lower than as recorded in the paper Exploring the Limitations of Autonomous Driving. Am I using the same checkpoint as was used in the paper? Are there any reasons that my results on the same experiment would be lower?