HumanCompatibleAI / adversarial-policies

Find best-response to a fixed policy in multi-agent RL
MIT License
275 stars 47 forks source link

Density Modeling of Victim Policy Activations #15

Closed AdamGleave closed 5 years ago

AdamGleave commented 5 years ago

Adds a pipeline to generate activations of a victim policy network across a range of opponents, fit a Gaussian Mixture Model and plot a bar chart and other visualizations of the mean log likelihood of the activations. Adds test cases for this & the t-SNE pipeline. Some other minor tweaks to t-SNE plotting.

As usual, anything affecting training or scoring should be scrutinized closely. Density model fitting I expect us to use a fair bit so should be looked at reasonably closely, but it'll only be as an interactive tool where errors should be obvious so not as important. Plotting code lowest standards.

codecov[bot] commented 5 years ago

Codecov Report

Merging #15 into master will increase coverage by 5.12%. The diff coverage is 37.09%.

Impacted file tree graph

@@            Coverage Diff            @@
##           master     #15      +/-   ##
=========================================
+ Coverage   53.07%   58.2%   +5.12%     
=========================================
  Files          54      57       +3     
  Lines        4516    4883     +367     
=========================================
+ Hits         2397    2842     +445     
+ Misses       2119    2041      -78
Flag Coverage Δ
#aprl 10.87% <0%> (-0.89%) :arrow_down:
#modelfree 51.21% <37.09%> (+5.69%) :arrow_up:
Impacted Files Coverage Δ
src/modelfree/visualize/styles.py 100% <ø> (+100%) :arrow_up:
src/modelfree/multi/common.py 71.13% <ø> (ø) :arrow_up:
src/modelfree/policies/transparent.py 75% <ø> (ø) :arrow_up:
src/modelfree/density/visualize.py 0% <0%> (ø)
src/modelfree/visualize/util.py 0% <0%> (ø) :arrow_up:
src/modelfree/visualize/training.py 0% <0%> (ø) :arrow_up:
src/modelfree/tsne/pipeline.py 90.24% <100%> (+90.24%) :arrow_up:
src/modelfree/configs/multi/score.py 44.06% <100%> (+22.3%) :arrow_up:
src/modelfree/density/fit_density.py 44.88% <44.88%> (ø)
src/modelfree/tsne/visualize.py 87.09% <69.23%> (+87.09%) :arrow_up:
... and 17 more

Continue to review full report at Codecov.

Legend - Click here to learn more Δ = absolute <relative> (impact), ø = not affected, ? = missing data Powered by Codecov. Last update 55be101...938dfe3. Read the comment docs.

AdamGleave commented 5 years ago

@decodyng I left some TODOs in fit_density.py that I didn't have enough context to resolve, you may have thoughts on (e.g. if you've used these features in some code not checked in to this branch)