huawei-noah / SMARTS

Scalable Multi-Agent RL Training School for Autonomous Driving
MIT License
940 stars 190 forks source link

Ensure performance meets internal benchmark #47

Open Gamenot opened 3 years ago

Gamenot commented 3 years ago

Benchmark,

1 agent, 50 SVs, loop scenario: 100 FPS 10 agents, 0 SVs, loop scenario, 25 FPS

Feel free to suggest a different benchmark than ^

Context:

davidrusu: Trying to run both agents and social vehicles together hides the performance hit from each individually. Here's the curves of how increasing either value, slows our performance: image

Tests were run on the loop scenario and Agents were instances of the following AgentSpec:


class Policy(AgentPolicy):
def act(self, obs):
return "keep_lane"

agent_spec = AgentSpec( interface=AgentInterface.from_type(AgentType.Laner, max_episode_steps=1000), policy_builder=Policy, )


> davidrusu: @jun Well, we can't directly control how many social vehicles are in the scenario, but here's a few episodes with a traffic flow of 30SV/min + 10 agents.

╭────────────────────┬────────────────────┬────────────────────┬────────────────────┬────────────────────┬╮ │ Episode │ Sim T / Wall T │ Total Steps │ Steps / Sec │ Scenario Map ││ ├────────────────────┼────────────────────┼────────────────────┼────────────────────┼────────────────────┼┤ │ 0/100 │ 2.18 │ 999 │ 21.78 │ loop ││ │ 1/100 │ 1.32 │ 999 │ 13.23 │ loop ││ │ 2/100 │ 1.24 │ 999 │ 12.35 │ loop ││ │ 3/100 │ 1.59 │ 999 │ 15.87 │ loop ││

wuyukun-tong commented 2 years ago

![Uploading image.png…]() 运行vis.py的时候出现了这个错误是怎么回事,麻烦大神解答一下,谢谢 Successfully registered the following environments: ['MetaDrive-test-v0', 'MetaDrive-validation-v0', 'MetaDrive-v0', 'MetaDrive-10envs-v0', 'MetaDrive-1000envs-v0', 'MetaDrive-training0-v0', 'MetaDrive-training1-v0', 'MetaDrive-training2-v0']. Traceback (most recent call last): File "vis.py", line 83, in env.render(mode="top_down", num_stack=25) File "/home/gzz/miniconda3/envs/copo/lib/python3.7/site-packages/metadrive/envs/base_env.py", line 265, in render return self._render_topdown(*args, *kwargs) File "/home/gzz/miniconda3/envs/copo/lib/python3.7/site-packages/metadrive/envs/base_env.py", line 480, in _render_topdown self._top_down_renderer = TopDownRenderer(args, **kwargs) File "/home/gzz/miniconda3/envs/copo/lib/python3.7/site-packages/metadrive/obs/top_down_renderer.py", line 143, in init assert self.current_track_vehicle is not None, "Specify which vehicle to track" File "/home/gzz/miniconda3/envs/copo/lib/python3.7/site-packages/metadrive/obs/top_down_renderer.py", line 235, in current_track_vehicle return self.engine.agent_manager.active_agents["default_agent"]