search

google-deepmind / meltingpot

A suite of test scenarios for multi-agent reinforcement learning.
Apache License 2.0
577 stars 116 forks source link

Field of view visualisation #167

Closed olipinski closed 1 year ago

olipinski commented 1 year ago

Hi!

I am working to replicate the results from the Hidden Agenda paper, and was wondering if there was the best way to add field of view visualisation to the renders? I think this would be very useful for evaluation, to see what the agents can focus on.

Currently, my best guess is to modify the WORLD.RGB observation to add borders or a tint to the agents' FOV. Is there a better way?

Thanks!

duenez commented 1 year ago

Hello,

I'm not sure I completely understand what you mean. Do you mean that you want to see what agents can see, but only on the "WORLD.RGB" observation, and not for learning? If that's the case, you could achieve this with direct post processing. Since you can extract position and orientation of all agents (using debug observations, disabled by default), you can then take the WORLD.RGB and add borders or tint, like you suggest.

At some point we were thinking about implementing a view system that would respect walls (i.e. not being able to see beyond a wall bend), but I don't think that's what you are talking about, right?

olipinski commented 1 year ago

Yes, I would like to be able to highlight what the agents can see on the WORLD.RBG observation, similar to the figure below.

example

Thank you for the pointers with the debug observations! I was hoping there would be an easier way to do this, to not slow down the simulation by having more observations being added.

And yes, the walls are not an issue for me.

Thanks you!