There are now many more benchmarking notebooks.
I tried to uniformize them during their development (also because now the new protopipe-BENCHMARK script requires them to behave and interface with the benchmarks.yaml configuration file in the same way).
Nonetheless, some of them still have some slight differences and possible small bugs (see e.g. #173).
This effort should also take into account more and more refactoring of code from the notebooks to the "new" API under protopipe.benchmarks.
Being this a long work, it cannot be tight to the release of v0.5.0...
The repository should provide a template notebook to avoid new notebooks to diverge from the expected behavior and also to ease the update/fixes of existing notebooks.
There are now many more benchmarking notebooks. I tried to uniformize them during their development (also because now the new
protopipe-BENCHMARKscript requires them to behave and interface with thebenchmarks.yamlconfiguration file in the same way).Nonetheless, some of them still have some slight differences and possible small bugs (see e.g. #173). This effort should also take into account more and more refactoring of code from the notebooks to the "new" API under
protopipe.benchmarks. Being this a long work, it cannot be tight to the release of v0.5.0...The repository should provide a template notebook to avoid new notebooks to diverge from the expected behavior and also to ease the update/fixes of existing notebooks.