Mathics3 / mathics-benchmark

Benchmark tools for mathics-core
https://github.com/Mathics3/mathics-core
GNU General Public License v3.0
2 stars 0 forks source link

Store the significative differences in a text file #22

Open mmatera opened 3 years ago

mmatera commented 3 years ago

Something that would be useful would be, after running the several benchmarks, to store in a text file which of those tests have presented significant differences (>%1).

rocky commented 3 years ago

Actually, I had thought about this too. What I currently do is run with the verbose option -v and work with the output printed. As a short-term workaround you can redirect the stdout to a file.

But yes, some sort of filtered data as text would be nice too.

Related to this is reordering the lines so the biggest gains are at the top.

There are all sorts of display filtering options one might imagine. Like show only the top/bottom by either a percent cutoff or an absolute number.