search

assume-framework / assume

ASSUME - Agent-based Simulation for Studying and Understanding Market Evolution
https://assume.readthedocs.io
20 stars 5 forks source link

Rl save policies debugging #349

Closed kim-mskw closed 1 month ago

kim-mskw commented 2 months ago

so the old save policies function did not really use max_eval. It was always set back to -1000 when the learning role was wirtten new. I changed that now and added an early stopping criterium. Something works better but something weird as well, because the max avg metrik is now always 19.28

This is the basis for the discussion with @nick-harder ick- tomorrow

kim-mskw commented 2 months ago

okay I know why it is always the same, because it somehow now always loads old policies in a dead folder. I am fixing that. This is terribly confusing.

kim-mskw commented 2 months ago

so there was a function in the learning_strategies that loaded policies and one in the learning_algo, leading to very weird behaviour after the load_scenario function. Basically in the eval run always an old policy from a folder was read.

Technically we need the actor in the bidding strategy, because otherwise it is not initialized if we do not use learning, so learning mode off and hence no algo as well. This double initializing is weird though.

codecov[bot] commented 1 month ago

Codecov Report

Attention: Patch coverage is 78.78788% with 7 lines in your changes are missing coverage. Please review.

Project coverage is 78.07%. Comparing base (ef8da6f) to head (3e997f1).

Files Patch % Lines
assume/reinforcement_learning/learning_role.py 53.84% 6 Missing :warning:
assume/scenario/loader_csv.py 94.11% 1 Missing :warning:
Additional details and impacted files ```diff @@ Coverage Diff @@ ## main #349 +/- ## ========================================== - Coverage 78.11% 78.07% -0.05% ========================================== Files 43 43 Lines 5068 5090 +22 ========================================== + Hits 3959 3974 +15 - Misses 1109 1116 +7 ``` | [Flag](https://app.codecov.io/gh/assume-framework/assume/pull/349/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=assume-framework) | Coverage Δ | | |---|---|---| | [pytest](https://app.codecov.io/gh/assume-framework/assume/pull/349/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=assume-framework) | `78.07% <78.78%> (-0.05%)` | :arrow_down: | Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=assume-framework#carryforward-flags-in-the-pull-request-comment) to find out more.

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.