search

mila-iqia / atari-representation-learning

Code for "Unsupervised State Representation Learning in Atari"
https://arxiv.org/abs/1906.08226
MIT License
240 stars 50 forks source link

Compute Orginal Paper's Metrics: Categorywise Average #42

Closed eracah closed 5 years ago

eracah commented 5 years ago

This pull request will:

  1. compute the average for each category at run time for a given method and game combo
  2. average these category averages together to create an overall score

The following items are added to acc_dict and f1_dict:

  1. "overall_avg": just the average over all probes
  2. "_avg" : for example "agent_localization_avg", which is the average accuracy of every state variable, which is considered an agent_localization state variable
  3. "across_categories_avg": the average of all category averages