Add ability to compare different provers

sneeuwballen / benchpress

Testing and benchmarking tool for logic-related programs.

BSD 2-Clause "Simplified" License

26 stars 8 forks source link

Add ability to compare different provers #63

Closed bclement-ocp closed 1 year ago

bclement-ocp commented 1 year ago

There is currently a /compare/ endpoint, but sometimes it can be nice/useful to compare two different provers from two different runs.

This PR provides a new endpoint /compare2/ available from the home page that allows to select two arbitrary provers from any two runs and compare them.

c-cube commented 1 year ago

Would you happen to have a screenshot of what the new /compare2/ page looks like? :) Otherwise, this looks very useful!

bclement-ocp commented 1 year ago

It's very basic and just re-uses the same table as the existing /compare/ page (but without specifying old/new because they are in the selects --- although I guess it can get confusing if you change the select values without resubmitting):

The select are also very basic with an optgroup with each file ID:

The UX can probably be improved (and I also want to get a full comparison as in the "show table of results" option) but I think it already does its job.

c-cube commented 1 year ago

Oh yeah that's cool, thank you! Generating graphs could also be nice if they're not there.