huawei-noah / SMARTS

Scalable Multi-Agent RL Training School for Autonomous Driving
MIT License
956 stars 190 forks source link

[Feature request] Render pedestrians in camera observations #1992

Closed Gamenot closed 1 year ago

Gamenot commented 1 year ago

Motivation

Pedestrians currently show up in the low-dimension observations but not in the camera observations. There is a need to render them in the observations to allow for agents that only use cameras to be able to interact with pedestrians.

Proposal

Pedestrians should show up in observations. Pedestrians should be accented to show up more distinctly in low-resolution images if possible. Perhaps a pedestrian-specific colour?

This might be a useful point to change to update the renderer using the simulation frame instead of direct updates using per-vehicle vehicle states.

Alternatives

No response

Additional context

No response

saulfield commented 1 year ago

It looks like pedestrians do show up in the camera observations already (see below image). Any thoughts on what enhancements we should make to them (if any)? Perhaps a distinct colour, size increase, or some other visual indicator added to them, like a circle?

{ Type Agent_0-Top-Down RGB }(3)