glopezdiest
commented
1 year ago No, the leaderboard cannot run asynchronously. While the calculation of the agent is indeed an interesting consideration in theory, there are just too many variables to have that as an interesting metric.
On one hand, some of the CARLA processes need the synchronous mode to ensure a correct behavior, such as the behavior of the TrafficManager, among others, which is why the LB can't run asynchronously. This could be fixed with time and effort though.
On the other hand, what about the different processes of the simulation? Physics / Scene calculations are quite heavy processes that will probably won't make the server go in real time, affecting the simulation-agent asynchronous comunication. Timewise, sensor rendering might work differently than real time ones, should that be considered as part of the agent's calculation time? Lastly, the hardware makes a huge impact too, and I'm not sure if that would be a good variable to take into account
I have known that leaderboard runs in synchronization mode.However,I wonder that if it can run in asynchronous mode since I think that the calculation time of an agent should be taken into consideration.