Chapter 4 Implementation

[sritchie]

This should get all the chapter 4 dynamic programming graphs up and running.

Gridworld

• [x] do "NOT in place" policy evaluation
• [x] add support for terminal states to GridWorld... and states in general?
• [x] add a "policy stable?" check, that checks if the best action is the same across all states.
• [x] I think we need to do all of this for a categorical state, that gives us access to dynamics.
• [x] take THAT code and make the Id version that we had before using mapK.

[codecov[bot]]

Codecov Report

Merging #25 into develop will increase coverage by 29.92%. The diff coverage is 59.79%.

@@             Coverage Diff              @@
##           develop      #25       +/-   ##
============================================
+ Coverage     5.42%   35.34%   +29.92%     
============================================
  Files           26       29        +3     
  Lines          479      529       +50     
  Branches        15       23        +8     
============================================
+ Hits            26      187      +161     
+ Misses         453      342      -111

Impacted Files	Coverage Δ
...rc/main/scala/io/samritchie/rl/policy/Random.scala	66.66% <ø> (+66.66%)	:arrow_up:
...ot/src/main/scala/io/samritchie/rl/plot/Plot.scala	0% <ø> (ø)
...c/main/scala/io/samritchie/rl/plot/Tabulator.scala	0% <ø> (ø)
...rc/main/scala/io/samritchie/rl/book/Chapter2.scala	0% <ø> (ø)	:arrow_up:
...rc/main/scala/io/samritchie/rl/policy/Frozen.scala	0% <0%> (ø)	:arrow_up:
...l-core/src/main/scala/io/samritchie/rl/State.scala	66.66% <0%> (+66.66%)	:arrow_up:
...in/scala/io/samritchie/rl/value/Contramapped.scala	0% <0%> (ø)
...rc/main/scala/io/samritchie/rl/book/Chapter4.scala	0% <0%> (ø)
...-core/src/main/scala/io/samritchie/rl/Policy.scala	0% <0%> (ø)	:arrow_up:
...rc/main/scala/io/samritchie/rl/book/Chapter5.scala	0% <0%> (ø)
... and 20 more

---

Continue to review full report at Codecov.

Legend - Click here to learn more Δ = absolute <relative> (impact), ø = not affected, ? = missing data Powered by Codecov. Last update bf62baf...b27fbe7. Read the comment docs.