search

Benchmarking-Initiative / Benchmark-Models-PEtab

A collection of mathematical models with experimental data in the PEtab format as benchmark problems in order to evaluate new and existing methodologies for data-based modelling
BSD 3-Clause "New" or "Revised" License
26 stars 14 forks source link

Adding suggestion of optimizers for benchmark models. #145

Open stephanmg opened 2 years ago

stephanmg commented 10 months ago

Data needs to be updated with final results.

dilpath commented 2 months ago

Thanks, this will be great to have!

I think for maintainability, a TSV table with the "raw" data is preferable. I can add a script that converts it to the markdown table and embeds it into the README. That way contributors can simply edit the TSV table with easy-to-understand "raw data" that they probably have like optimizer, number of starts, number of successful starts, time per start / whatever else, and the script can then compute the overall efficiency if sufficient data is provided.

We would also need a few more details about the optimizer and their hyperparameters (e.g., what is nlopt_42?). Some TSV with columns optimizer_id and description would suffice, we can then think how to present it...