Add incremental benchmarking scripts

jerhard commented 1 year ago

This PR adds python scripts for benchmarking the incremental analysis. These scripts were originally added by @stilscher in https://github.com/goblint/analyzer/pull/778. In the discussion there, we decided to move the scripts to the bench repo.

Compared to #778 the following changes were made:

Project specific settings are to be defined in projects.py
a configuration for sqlite was added
Project name is to be passed as a command line argument for the scripts
the command that goblint is started with and the commit-id of the analyzed repo are recorded in an output-file

Closes #47 Closes #48

TODO

[x] Confs from https://github.com/goblint/analyzer/pull/936.

michael-schwarz commented 5 months ago

@stilscher: I assume you simply forgot about the review request here?

Unless there are objections, I would propose to merge this.

sim642 commented 5 months ago

At minimum, one should check if the confs are still valid for Goblint's schema.

goblint / bench

Add incremental benchmarking scripts #44

TODO