Classic control benchmark

dekuenstle commented 5 years ago

Changes:

Predefine reward thresholds of convergence criteria for the particular environment
Use max episode timesteps predefined by the particular environment
Use mean of episode rewards as run performance
Log mean performance every 10th episode
Command line interface for running default configuration
Stop at maximum episodes OR maximum timesteps

KEggensperger commented 5 years ago

Are you still working on this? Just ping me when this is ready to be reviewed.

dekuenstle commented 5 years ago

Are you still working on this? Just ping me when this is ready to be reviewed. @KEggensperger I just resolved an issue with installation and added a useful feature. Work on this should be done for now.

dekuenstle commented 4 years ago

This looks great! Before merging, I have 3 really minor requests:

Could you please add a version string 0.2 (as in https://github.com/automl/HPOlib2/blob/development/hpolib/benchmarks/ml/autosklearn_benchmark.py#L25)

Add your changelog (the text of this PR) as a comment in the code (e.g. as a note in the meta-dictionary) so we know why this has been changed

Remove the command line interface (no other benchmark has this). If you think the code is particular useful, feel free to move it to the example directory

I will then take care of this benchmark and see whether I can rebase this branch. Thanks a lot!

Thank you for the review! I changed according to your suggestions.

automl / HPOlib1.5

Classic control benchmark #74