search

HumanCompatibleAI / adversarial-policies

Find best-response to a fixed policy in multi-agent RL
MIT License
275 stars 47 forks source link

CI improvements: more granular code coverage, parallelism on local machines #31

Closed AdamGleave closed 4 years ago

AdamGleave commented 4 years ago

Breakdown code coverage based on if it is a core part of the codebase, auxiliary (beta code, configs) or tests. These have different thresholds in the coverage checks.

Also supports parallelism on local machines. We can't turn this on for Travis due to small resources in VM, but may be able to enable it later if we get beefier VMs.

codecov[bot] commented 4 years ago

Codecov Report

Merging #31 into master will increase coverage by 2.03%. The diff coverage is 0%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master      #31      +/-   ##
==========================================
+ Coverage    58.4%   60.44%   +2.03%     
==========================================
  Files          58       63       +5     
  Lines        4921     5225     +304     
==========================================
+ Hits         2874     3158     +284     
- Misses       2047     2067      +20
Flag Coverage Δ
#aprl 13.64% <0%> (+2.85%) :arrow_up:
#modelfree 51.27% <0%> (-0.21%) :arrow_down:
Impacted Files Coverage Δ
src/aprl/beta/self_play.py 0% <ø> (ø)
tests/aprl/test_agents.py 98.97% <ø> (ø)
src/aprl/agents/__init__.py 100% <ø> (ø) :arrow_up:
src/aprl/train_self_play.py 0% <0%> (ø) :arrow_up:
src/aprl/beta/ppo_self_play.py 0% <0%> (ø)
tests/modelfree/test_envs.py 100% <0%> (ø)
tests/modelfree/test_experiments.py 97.75% <0%> (ø)
tests/aprl/test_envs.py 94.11% <0%> (ø)
tests/aprl/test_common.py 100% <0%> (ø)
... and 1 more

Continue to review full report at Codecov.

Legend - Click here to learn more Δ = absolute <relative> (impact), ø = not affected, ? = missing data Powered by Codecov. Last update 8f7fc26...a456ab1. Read the comment docs.

AdamGleave commented 4 years ago

The different codecov status checks aren't being generated and I can't figure out why. I think codecov may be having some problems. The status checks disappeared from evaluating-rewards previously when they used to work (and config hasn't changed).

AdamGleave commented 4 years ago

Although the codecov status checks seem to still not be working, I think this is a codecov bug, so am going to go ahead and merge and revisit if codecov does not fix it soon.