aspuru-guzik-group / olympus

Olympus: a benchmarking framework for noisy optimization and experiment planning
https://aspuru-guzik-group.github.io/olympus/
MIT License
81 stars 22 forks source link

[Dev] Results from `Olympus().run()` contains results from previous runs #29

Open wiseodd opened 1 year ago

wiseodd commented 1 year ago

An issue with Olympus ’s run (and thus benchmark) method:

oly = Olympus()

for dataset in datasets:
    oly.run(dataset, ...)
    values = oly.campaign.observations.get_values().flatten()

When looking at the values of dataset[i] , it will contain values obtained from dataset[0..i-1] , so the returned value list becomes longer and longer.

I managed to circumvent this by doing the following and not using Olympus().benchmark.

for dataset in datasets:
    campaign = Campaign()
    Olympus().run(dataset, ..., campaign=campaign)
    values = campaign.observations.get_values().flatten()