Unified Graphics Dashboard

[nathan-miller23]

Creating a central, interactive GUI for all forms of overcooked visualization

[bmielnicki]

Example list of desired features for dashboard:

• loading in existing agents and allowing users to play human-AI or AI-AI games (basically Overcooked-Demo as it is now)
• use of state visualizations to watch existing trajectories
• visualizations to better understand policies as they train (i.e. % useful of onion pickups, % optimal tomato pottings over time)
• visualizations to better understand policies once they have been trained (i.e a heatmap to show how "on-distribution" [how frequented it is in rollouts] a state is under the current policy, visualizing the entropy as a function of state, etc)
• flexibility to new additions

[bmielnicki]

One of the most important design decisions here is if we want to use python for GUI or javascript that can be runnable from python.

Pygame offers enough features to support https://github.com/HumanCompatibleAI/overcooked_ai/issues/45 and some GUI interactivity.

How pygame solution pros and cons look like:

• matplotlib charts can be put inside pygame window
• interactivity in charts is for sure possible given that pygame supports mouse events
  • it can take way more development time to do something interactive than in javascript
• there is possible to export pygame image to file so it can be viewed inside ipython cells
  • although it would be not interactive, interactivity for ipython cells is probably not the top priority
• Running interactive pygame window needs to be run in loop that listens to events (so python process is blocked until the window is not closed)

How javascript solution pros and cons look like:

• work interactively in both browser and ipython cell (this code is already made for visualizations)
• visualization of the state is already written in overcooked_demo
  • less code to maintain if we want to keep overcooked_demo in current js form
  • no time (or at least less time to change existing code instead of writing a new one from scratch) spent for development pygame visualization of state and chart in the pull request: https://github.com/HumanCompatibleAI/overcooked_ai/pull/38
• less elegant in terms of code simplicity
  • additional programming language
  • creating HTML/js using strings operations
  • once HTML runs it can only communicate with python through files (or though Flask webserver)

I'm a bit more inclined toward javascript solution, but I can imagine why reducing javascript code to 0 can be more useful.

@nathan-miller23 and @micahcarroll - let me know what do you think about that because https://github.com/HumanCompatibleAI/overcooked_ai/pull/38 and https://github.com/HumanCompatibleAI/overcooked_ai/issues/45 heavily depend on this design decision.

[nathan-miller23]

Ok I'm really on the fence about this one actually. I agree that the JS solution is easier (this is a task the programming language was specifically designed for after all) and a lot of time and dev effort has gone into Overcooked-Demo, so it would be a shame to re-build the exact same functionality in python. At the same time, the codebase has become increasingly streamlined and clean the more we've migrated away from JS and to python.

Taking a step back, my high level goal for this "dashboard" was to create a tensorboard-like tool, where the user could run something like overcooked-visualization --logdir . and open an interactive GUI where they can view charts, trajectories, training data, and play real-time games either human-human, AI-AI or human-AI. That task seems more conducive of using JS.

The more I think on it, the more I'm inclined to say we should be doing this project inside Overcooked-Demo, where we use Overcooked-ai as a dependency and stick to the paradigm of offloading as much of the complexity to the server-side python code as possible. Then, the JS code becomes minimalistic, and is basically just a last second wrapper to convert processed states provided by python into graphics, and as a client event listener.

I still think the native python visualization (for converting states or trajectories into pretty graphics for use in Jupyter notebooks) is super useful, and is worthwhile to pursue in addition. Maybe that means writing a python wrapper on Overcooked-Demo so that all functionality can also be done programmatically from the command line or something. For example, loading agent A and agent B and starting an AI-AI game could look like demo.py game --agent-0 /path/to/A --agent-1 /path/to/B and that spawns a browser window with the game. This might be unnecessary complicated due to the fact that Overcooked-Demo is Dockerized however.

I'm curious to hear @micahcarroll input, as he has more experience tabulating and gathering results from experiments and might have valuable insight into what visualizations/workflows are practical and useful

[micahcarroll]

Thanks so much Bartek for the really good breakdown of pros and cons!

Thinking about the choice, I think that the main deal-breaker here is that we gather human data through a JS implementation of the game: one of the things that I've found most useful in debugging agents that are related to human data is having the way we evaluate / study / play with trained agents be consistent with how the human data collection has actually happened (and similarly "how human evaluation will happen in future"). This is useful because you can better reason about what things went wrong at data-collection time or what things will go wrong at evaluation time.

Given that the human data collection interface that humans interface with is in fact in Javascript (and I think would be hard / non-smooth to deliver in python, especially because psiturk requires javascript anyways + Nathan just wrote a bunch of code for this recently, so doesn't seem to make sense to overturn that), I end up concluding the same thing as Nathan: we should go with the JS approach, but try to have as much of the dashboard logic in python as possible, basically just adding onto the overcooked-demo repo.

But yes, I also agree that having a native python visualization framework is still useful. This could enable investigating training via pixels (rather than state representations) – as it would require creating visualizations very quickly (which is basically impossible if python has to communicate with javascript to get the visualizations), and would offer a useful & easy way to visualize trajectories in jupyter notebooks, terminals with image support (iterm2 can do this), and potentially other things too? I'm not sure if people have other reasons to want this too, but for me, jupyter visualization support would have come in handy a lot in the past.

One thing that you mentioned (the "heatmap" of states) is quite cool: I did some preliminary investigation here (don't worry about not being able to access the link – it is mainly a pointer to myself, and this code doesn't actually work) in using t-sne, and think that it might be a cool thing to look into more in the future.

[bmielnicki]

Thanks Nathan and Micah for the replies.

Based on them I try to break down this task into smaller ones (probably each one can be separate pull request):

1. Writing python wrappers for overcooked-demo code to make everything runnable from the command line. I'm not very familiar with this library so I assume that @nathan-miller23 can help me by listing what would be beneficial to have wrapped.

2. Create the first version of the javascript dashboard that uses all existing, relevant code - https://github.com/HumanCompatibleAI/overcooked_ai/pull/38 and result of 1. All runnable from the command line or other python code. It would work as "python visualization_dashboard.py --trajs-dir=trajectories_directory --agents-dir=agents_directory".

3. Improve code https://github.com/HumanCompatibleAI/overcooked_ai/pull/38 to better visualize single trajectory (add usage of adjectives, distinguishing in visualization between item drop, potting, and soup delivery)

4. Add multi trajectories charts to demonstrate progress in training (one point in the chart is one trajectory) and at them to dashboard:

   • % of event adjectives through trajectories
   • the absolute number of events and its adjectives

Given that Micah already started working on heatmaps I will leave this part to him.

But yes, I also agree that having a native python visualization framework is still useful. This could enable investigating training via pixels (rather than state representations) – as it would require creating visualizations very quickly (which is basically impossible if python has to communicate with javascript to get the visualizations), and would offer a useful & easy way to visualize trajectories in jupyter notebooks, terminals with image support (iterm2 can do this), and potentially other things too? I'm not sure if people have other reasons to want this too, but for me, jupyter visualization support would have come in handy a lot in the past.

Create an image from the state purely in python is probably independent on unified GUI task if we have already js code that visualizes state in overcooked-demo. I already have like half of the work here done by tinkering with pygame to learn how it works so I will probably do it anyway before starting this task. I can add an optional function parameter to visualize an image in a jupyter cell (displaying image is one line of code more).

Let me know if there is something more you want to be done besides subtasks above or if I misunderstood any part of this task.

[micahcarroll]

Yes, sorry I was unclear – the main thing I was trying to say (echoing Nathan's comment) is that I believe that #45 is still useful, even though we have decided not to use it for the Unified Graphics Dashboard (if we had chosen to do the unified dashboard in python, then #45 would have been used for this too).

[bmielnicki]

PR for some stuff mentioned here: https://github.com/HumanCompatibleAI/overcooked-demo/pull/30 .

Maybe that means writing a python wrapper on Overcooked-Demo so that all functionality can also be done programmatically from the command line or something. For example, loading agent A and agent B and starting an AI-AI game could look like demo.py game --agent-0 /path/to/A --agent-1 /path/to/B and that spawns a browser window with the game. This might be unnecessary complicated due to the fact that Overcooked-Demo is Dockerized however.

I'm not sure how to do in the current form as it is dockerized (except running up.sh with params that sets up what is initially displayed). It is also likely that most of the things (like make AI-AI game) is possible to do using python script without referencing overcooked-demo at all.

4. Add multi trajectories charts to demonstrate progress in training (one point in the chart is one trajectory) and at them to dashboard:

• % of event adjectives through trajectories
• the absolute number of events and its adjectives

It is non-obvious how to handle it inside overcooked-demo GUI as it would require to select multiple trajectories while having them ordered in sequence. I will wait with doing that for now but if you want to have this task done let me know.