HumanCompatibleAI / evaluating-rewards

Library to compare and evaluate reward functions
https://arxiv.org/abs/2006.13900
Apache License 2.0
61 stars 7 forks source link

Add `scripts.distances.npec` to perform NPEC comparisons in parallel using Ray #44

Closed AdamGleave closed 3 years ago

AdamGleave commented 3 years ago
codecov[bot] commented 3 years ago

Codecov Report

Merging #44 into master will decrease coverage by 0.15%. The diff coverage is 73.74%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master      #44      +/-   ##
==========================================
- Coverage   85.25%   85.10%   -0.16%     
==========================================
  Files          65       65              
  Lines        4231     4262      +31     
==========================================
+ Hits         3607     3627      +20     
- Misses        624      635      +11     
Impacted Files Coverage Δ
.../evaluating_rewards/analysis/distances/heatmaps.py 97.01% <ø> (+6.60%) :arrow_up:
src/evaluating_rewards/distances/common_config.py 94.73% <ø> (+14.03%) :arrow_up:
...wards/analysis/distances/plot_gridworld_heatmap.py 85.95% <14.28%> (-3.62%) :arrow_down:
...luating_rewards/analysis/distances/plot_heatmap.py 66.46% <47.82%> (-10.23%) :arrow_down:
src/evaluating_rewards/analysis/results.py 33.02% <66.66%> (+6.40%) :arrow_up:
src/evaluating_rewards/scripts/distances/npec.py 79.68% <79.68%> (ø)
src/evaluating_rewards/scripts/distances/common.py 91.85% <83.58%> (-8.15%) :arrow_down:
src/evaluating_rewards/scripts/distances/epic.py 91.59% <100.00%> (-0.26%) :arrow_down:
src/evaluating_rewards/scripts/distances/erc.py 89.47% <100.00%> (ø)
tests/test_scripts.py 100.00% <100.00%> (ø)
... and 3 more

Continue to review full report at Codecov.

Legend - Click here to learn more Δ = absolute <relative> (impact), ø = not affected, ? = missing data Powered by Codecov. Last update 1d5ad28...2df9c0b. Read the comment docs.