search

mohanak12 / mb-unit

Automatically exported from code.google.com/p/mb-unit
0 stars 1 forks source link

feature request: automated performance monitoring #630

Open GoogleCodeExporter opened 9 years ago

GoogleCodeExporter commented 9 years ago 
I’d like to see Gallio write out an XML file during each test run and
record the execution times for each test. Then, at the next run, I’d like
Gallio to list the new execution time next to the old and the absolute or %
difference which I can then sort by. I’d also like to be able to configure
thresholds to classify a test as failed, if the new time is so many seconds
or so many % slower than in the previous run.
The XML file would need to include also the PC hardware configuration and
make sure it still matches, so that we are comparing apples to apples.

If this is already possible to do, please tell me how. If not, can it be
added easily?  Does anyone have time to do so?  If possible, but no one has
time, how do I best go about adding it myself?

Thanks,
Ingmar

Original issue reported on code.google.com by ingmar.b...@gmail.com on 4 Mar 2010 at 6:27

GoogleCodeExporter commented 9 years ago 
Which test runner would you like this feature to appear in?  Icarus?

It's definitely possible to extract the relevant information from the XML report
files and build a table like this.

The long term plan has been to publish reports to a service called Archimedes 
which
would then be able to aggregate and compare test runs over time across the 
whole team.

Original comment by jeff.br...@gmail.com on 4 Mar 2010 at 7:19

GoogleCodeExporter commented 9 years ago 
Yes, We are using the Icarus test runner. It already reports the times of the 
current
run in the result table. So adding the other columns is I hope not too much to 
ask ;-)

Original comment by ingmar.b...@gmail.com on 4 Mar 2010 at 11:04

GoogleCodeExporter commented 9 years ago 
Seems pretty straightforward, especially in Icarus.

Original comment by jeff.br...@gmail.com on 5 Mar 2010 at 12:41

GoogleCodeExporter commented 9 years ago 
It doesn't keep track of previous runs (currently), but it wouldn't be too hard 
to
add the % change. The thresholds and hardware configuration are a bit more 
tricky.

Original comment by grahamr...@gmail.com on 5 Mar 2010 at 9:17

GoogleCodeExporter commented 9 years ago 
Don't worry about hardware configurations and stuff.  That makes more sense for 
an
external tool like Archimedes.

Original comment by jeff.br...@gmail.com on 5 Mar 2010 at 5:51

GoogleCodeExporter commented 9 years ago

Original comment by Yann.Tre...@gmail.com on 17 Jul 2010 at 8:20

GoogleCodeExporter commented 9 years ago

Original comment by Yann.Tre...@gmail.com on 14 Jun 2011 at 5:53