huawei-noah / SMARTS

Scalable Multi-Agent RL Training School for Autonomous Driving
MIT License
925 stars 186 forks source link

Render traffic lights in camera observations #2040

Closed saulfield closed 1 year ago

saulfield commented 1 year ago

Closes #1994.

traffic_light_obs

Scenario used for testing: 4lane_t_signals.zip

qianyi-sun commented 1 year ago

Good work! My only question now is if we should change the RGB colour of the ego vehicle.

@Adaickalavan

Then everything regards ego agent color in SMARTS has to be changed, like pictures and etc, and maybe the models have to be retrained as well.

Maybe we could consider to change the color of red light instead, like adding a color in RGB like trafficlight_red=(254,255,255), visually there shouldnt be much difference compared to (255,255,255), but code-wise there's difference and we dont have to touch all the current ego-agent-related stuff at all.

@Gamenot

Adaickalavan commented 1 year ago

Vision based policies will have difficulty distinguishing between similarly coloured objects, such as the red-coloured ego and red traffic light.

All existing documents and example gifs uses red to represent the ego. Therefore, to minimise any change needed, we may consider changing the red traffic light colour to a different shade of red such as maroon (128, 0, 0) which has non-negligible difference in rgb value compared to that of red (210, 30, 30).

Maroon-traffic-light

If the red traffic light colour is changed in the rgb image, similar change will be needed in Envision.

saulfield commented 1 year ago

I have changed the color of the stop signal to burgundy.

{ Type Agent_0-Top-Down RGB }

Adaickalavan commented 1 year ago

Internet search claims the following rgb values:

I had preferred Maroon since it is further away from our current Red = (210, 30, 30).

Our current "Burgundy" = (127, 0, 1) seems to be actually closer to Maroon and thus probably mislabelled. Our current "Burgundy" does not seem to be used anywhere in the code. Hence, we could consider rectifying the label and value.