Results of running a pretrained model is FAILURE

[hqli]

Hello, I followed the README.md and run a pretrained model. But the result is failure.

hardware

• i7-10700F CPU @ 2.90GHz,
• GTX3070(max fps is 13 with 3 cameras)

First Try

Service Start: ./CarlaUE4.sh -quality-level=Epic -world-port=2000 -resx=800 -resy=600 -opengl

Client Start:

export PORT=2000 
export ROUTES=leaderboard/data/routes_training/route_09.xml
export TEAM_AGENT=image_agent.py
export TEAM_CONFIG=../2020_CARLA_challenge_train_data/epoch=24.ckpt 
export HAS_DISPLAY=1

./run_agent.sh

Result

the car turned left at the first crossing, stopped in the yard by the side of the road(left turn failure). However I noticed that the car turned left at the second crossing when running Data Collection.

Second Try

I noticed that training data is 2Hz. I modify code leaderboard/autoagents/autonomous_agent.py.

import time first and modify __call__.

        # print('======[Agent] Wallclock_time = {} / {} / Sim_time = {} / {}x'.format(wallclock, wallclock_diff, timestamp, timestamp/(wallclock_diff+0.001))) 
        # 13frames/s -> 2frames/s  77ms -> 500ms  => diff 423ms
        time.sleep(0.423)

        control = self.run_step(input_data, timestamp)
        control.manual_gear_shift = False

        return control

I also add code for FPS show. And fps is 2 when run a pretrained model.

Result

the car turned left at the second crossing, but stopped in the yard by the side of the road((left turn failure)).

[hqli]

Thanks for your great work.

I also noticed [#39 ]. My Carla version also is 0.9.10.1.

This issues focus on that Carla service should start with 2 fps or not?

[bradyz]

2 Hz in this context does not mean the simulator should be modified to run at 2 Hz, but means we run the simulator at 20 FPS and collect data every 10 frames