Add infrastructure for offline pretraining of Bandits

[gkirgizov]

Will enable experiments with offline pretraining of models (both bandit and surrogate also) to compare with baseline GOLEM performance.

Changes:

• Provides infrastructure for collecting histories (HistoryReader) and training models separately from optimization (AgentTrainer). Those two can work in tnadem with ExperimentLauncher and ExperiementAnalyzer.
• Extends ExperienceBuffer to handle individual trajectories (instead of just steps).
• Extends experiment code in molecule_search/experiment.py to run with offline pretraining agent on histories.

Minor changes:

• Additionally, minor fix in MolAdapter & MolGraph
• Add __bool__ to Fitness for simpler checks

How to use offline training:

1. Collect histories to some directory using ExperimentLauncher
2. Create optimizer & Pretrain mutation agent on these histories using HistoryReader and AgentTrainer
3. Optionally, validate the Agent on validation set of histories
4. Run optimization with pretrained agent
5. Analyze results of experiments with ExperiementAnalyzer

Example in molecule_search/experiment.py

[aim-pep8-bot]

Hello @gkirgizov! Thanks for updating this PR. We checked the lines you've touched for PEP 8 issues, and found:

There are currently no PEP 8 issues detected in this Pull Request. Cheers! :beers:

Comment last updated at 2023-08-25 04:29:39 UTC

[gkirgizov]

@YamLyubov , please check commit Fix error with content loading of molecules in particular

[codecov-commenter]

Codecov Report

Merging #138 (b907105) into main (afa11e6) will decrease coverage by 1.71%. The diff coverage is 33.33%.

@@            Coverage Diff             @@
##             main     #138      +/-   ##
==========================================
- Coverage   73.84%   72.13%   -1.71%     
==========================================
  Files         131      135       +4     
  Lines        7841     8061     +220     
==========================================
+ Hits         5790     5815      +25     
- Misses       2051     2246     +195     

Files	Coverage Δ
golem/core/adapter/nx_adapter.py	83.33% <100.00%> (+1.66%)	:arrow_up:
golem/core/optimisers/adaptive/common_types.py	100.00% <100.00%> (ø)
...m/core/optimisers/adaptive/mab_agents/mab_agent.py	75.00% <100.00%> (ø)
...adaptive/mab_agents/neural_contextual_mab_agent.py	77.77% <100.00%> (ø)
golem/core/optimisers/fitness/fitness.py	86.90% <100.00%> (+0.31%)	:arrow_up:
golem/core/optimisers/genetic/evaluation.py	94.37% <100.00%> (ø)
...olem/core/optimisers/genetic/operators/mutation.py	89.21% <100.00%> (ø)
golem/core/optimisers/adaptive/operator_agent.py	88.73% <93.33%> (-3.51%)	:arrow_down:
golem/core/optimisers/adaptive/reward_agent.py	42.10% <0.00%> (-5.52%)	:arrow_down:
golem/core/optimisers/objective/objective.py	83.63% <50.00%> (-1.27%)	:arrow_down:
... and 6 more

... and 4 files with indirect coverage changes

[pep8speaks]

Hello @gkirgizov! Thanks for updating this PR. We checked the lines you've touched for PEP 8 issues, and found:

There are currently no PEP 8 issues detected in this Pull Request. Cheers! :beers:

Comment last updated at 2023-10-25 13:31:54 UTC