search

google-deepmind / dqn_zoo

DQN Zoo is a collection of reference implementations of reinforcement learning agents developed at DeepMind based on the Deep Q-Network (DQN) agent.
Apache License 2.0
457 stars 78 forks source link

Atari Result Summary Figure #12

Closed liziniu closed 3 years ago

liziniu commented 3 years ago

Hi,

First, thanks for sharing the results.tar.gz.

I am interested in the summary figure provided. However, I cannot exactly reproduce the curves with the given CSV files. My implementation is as follows: for each algorithm (e.g., DQN)，

  1. Compute the mean of normalized return over 5 random seeds for each environment.
  2. Plot the solid line with the median and the shaded region with maximal and minimal values over 57 environments using the results from Step 1.

I can basically reproduce the solid lines but the shaded region is not expected.

Your clarification would be very helpful!

Thanks Ziniu

jqdm commented 3 years ago

Thanks for your interest.

That is one way to generate a similar plot. The method used for the summary plot in the README.md was chosen so that it roughly follows the one in the Rainbow paper, but for multiple seeds.

To generate the summary plot, the data was first smoothed with a moving average with window of size 10 as mentioned in the README.md. Given an agent, frame and seed, the median normalized return over environment is calculated. Then the DataFrame was passed to seaborn.lineplot() with estimator=numpy.median and ci=100, so the shaded area captures the variation over seeds.

If that's not clear, a Colab notebook showing how the summary plot was generated will be uploaded soon.

jqdm commented 3 years ago

Added Colab notebook for generating plots in 98bd61d0b27f57a4f9cacc87002285ecf5d4c09e.