Duckietown - Fully Observable Navigation

[bhairavmehta95]

This has been a recent request from quite a few people, so I figured I'd give it its own little issue.

gym-duckietown is a quite unique tool in the landscape of simulation, and has many desirable properties of hard, reinforcement learning environments:

• it's a lot faster than many of the physics-based simulators out there
• the visual details are not trivial, but not too extensive to slow it down
• it's got a continuous action space, with the option for discrete actions (that make sense)
• it has the ability to be fully customized (first person vs overhead vs graph-based, obstacles to enable high level planning, reward function design)

One of the requests is that we make a mode for Duckietown to become a fully-observable environment, specifically from a top down view. This would allow for memory-less complex navigation tasks, in addition to the one that will be included in the AIDO release. In addition, since Duckietown itself is a "grid based" world, there has also been interest in making an easier version of the description above, using the traditional grid world physics (from some talks, this was described as useful to test out algorithms that succeed on a standard gridworld, without making too big of a jump to too hard on an environment).

The implementation would be simple enough; just change the camera position / tracking, using the camera settings from --draw-curve as a starting point.

Ideally, I'd like to develop this as a branch on my fork of the main repo, with the potential for merging if interest goes beyond myself + the few folks who've asked for something like this. Just more of a post to (maybe) garner interest, but also, as a heads up for people who come across this sometime in the future.

[maximecb]

Cool idea. You might be able to do better than just moving the camera and pointing it down as in the --draw-curves mode. There is a glOrtho function which you can use in place of gluLookAt. It's a camera mode without a field-of-view angle. It implements an orthographic projection, which means everything gets squashed down into a 2D image without perspective distortion.

http://thecodinguniverse.com/tag/glortho/ https://lmb.informatik.uni-freiburg.de/people/reisert/opengl/doc/glOrtho.html

[bhairavmehta95]

oh perfect! I will try this out, and keep progress updates posted here!

[bhairavmehta95]

@eracah You might want to put any specific feature requests here. Just so we can keep everything central.

[eracah]

ok cool thanks! will do!

[fgolemo]

@eracah Evan? How come you're working on this too? :D Got your fingers in everything.

[eracah]

haha. No actually, I just selfishly want some features for a project I am working on.

[eracah]

I think the main feature I'd like to see besides some version of "agent_pos" and "get_dir_vec" api from minigrid is an overhead view mask. That is, a (grid_size x grid_size) array where the value of each cell of the array corresponds to what is on the grid (0 for straight road, 1 for the car being in that grid, 2 for a stop sign, ... etc.) or something like that.

That'd be pretty cool! Let me know if that is doable. Thanks!

[maximecb]

some version of "agent_pos" and "get_dir_vec" api from minigrid

There are cur_pos and cur_angle fields which give you the position and direction of the agent. The position is tile-relative, and the angle is in radians, with angle zero pointing towards the +x axis.

[eracah]

ok perfect, I can discretize the angle to my liking. Thanks!

[maximecb]

For accessing the map there is a _get_tile(i, j) method which takes integer indices as input. It returns None if there is no tile at that position.

[bhairavmehta95]

This is the continuous action scenario. The gridworld-esque one will be similar.

[bhairavmehta95]

For the folks interested, a manual policy running in the gridworld. Not sure if we want to have the duckiebot centered in the tile (visually more appealing, but maybe some added randomization?)

[eracah]

This looks really good! Yeah would be cool to toggle between random and centered with a flag.

[bhairavmehta95]

https://github.com/bhairavmehta95/gym-duckietown/tree/gridworld-physics

It'll live in a branch (on a fork - so twice removed, poor guy :cry: ) for now, as its unclear how separate we'd like to keep this from the main simulator. I'm also figuring out how best to document / debug this (as issues on this repo, or the fork?) - unclear at the moment.

Two new modes have been added:

1. The main DuckietownEnv now comes with an aerial-view mode. Just changes the observation from First Person to what you see above. See manual_control.py in that branch on how to enable it.
2. Inside of gridworld.py, you can see how to enable the gridworld mode, which is exactly what is shown above. You can also enable first-person which gives you the same gridworld functionality, but without the aerial view. You can also enable start-centered to minimize the discrepancies between a gridworld environment, and this one.

In addition, with the gridworld environment, you now get masks inside of the info dictionary. The first array inside of info['masks'] is a one-hot encoding of where the agent is, and the second one is an array where the ones correspond to drivable tiles.

[maximecb]

I like the idea of keeping the masks in info because Gym and RL implementations have pretty poor support for complex observations that aren't just one n-dimensional tensor.

To decouple things, it might make some sense to have a hierarchy of Renderer classes that can be used to produce the observation and the human render. That way you could parameterize the resolution, anti-aliasing, the camera, etc. for different environments. Just an idea, but it might make some sense given the request in #85 (having a first-person observation but a top-view human render).

Sidenote: in my interior simulator, I'm going to show both a large human render and the low-res agent view in the corner (https://www.youtube.com/watch?v=bEPAc8B0YAA).

I think it makes sense to keep discussions in the main repo if you intend to merge it all back eventually. Otherwise it's two places people have to look.

[bhairavmehta95]

@maximecb I like the idea of having a hierarchy of renderers - I will sketch something up and maybe run it by this thread before implementing.

Also, is that code opensource yet?

[maximecb]

@bhairavmehta95 It's private for now just because there isn't very much to show, but I'm hoping to have a basic prototype in 3-4 weeks. I can give you early access if you're interested :)

[bhairavmehta95]

@maximecb I'm not sure if we ever wanted to merge this into master? Otherwise I will close this issue.

[maximecb]

Looks like it's already been merged. Seems fine to close the issue.

[bhairavmehta95]

I was more talking about the functionality of giving the agent the choice to do a gridworld vs get a top down view (the merged only gives the top down view on render rather than obs).

Anyway, if someone needs it, they'll probably stumble upon it. Closing now.

[pranoy-k]

Hi Bhairav, as @maximecb says these changes are merged to master, but I can't see aerial-view option in the main branch(manual_control.py). We are using this gym-environment to create a view from a pedestrian standpoint(the world view must be from pedestrian eyes), Is this possible with the current code, or do we have to undergo a feature implementation?

[maximecb]

You want to make an environment where pedestrians have to cross the street?

[bhairavmehta95]

You will need to add a feature implementation. Shouldn't be too hard, you just have to change the position of the camera. Please raise an issue for further discussion.

[pranoy-k]

@maximecb @bhairavmehta95 Thank you for the reply

@maximecb We want to control the pedestrians manually to let them cross the street given the autonomous car is roaming on the street.

Sure @bhairavmehta95 I will raise an issue, it makes more sense to discuss that there