search

mahmoudimus / nose-timer

A timer plugin for nosetests (how much time does every test take?)
MIT License
126 stars 33 forks source link

Provide a way for the plugin to dump a data file with timings #13

Closed mahmoudimus closed 9 years ago

mahmoudimus commented 11 years ago

The datafile can then be used by other 3rd party tools to track/graph slow tests over time

skudriashev commented 10 years ago

@mahmoudimus, what do you think would be the dump format? Thanks!

skudriashev commented 10 years ago

@mahmoudimus, any ideas? Thanks!

mahmoudimus commented 10 years ago

@skudriashev I have some ideas - think of how it would be in Jenkins to see which tests take an inordinate amount of time.

skudriashev commented 10 years ago

Can you, please, share your ideas? So we would implement this? Thanks!

e0ne commented 10 years ago

It will be great to do it in xunit format (--with-xunit option). All CI servers could work with it.

mahmoudimus commented 10 years ago

whoops. I agree with @e0ne

mahmoudimus commented 10 years ago

think of how something like coveralls (https://coveralls.io/) can be used to time our tests

skudriashev commented 10 years ago

@e0ne, look's like you can already use the --with-xunit option and it will infer tests names with timings. I don't see any reason to do the same, but for the timer plugin. Or I missing something?

skudriashev commented 10 years ago

Guys, what do you think?

aehlke commented 10 years ago

Personally I don't have a use for nose-timer after moving to --with-xunit. But that's just because I'm only reading the times by machine, not for human eyes. I don't see any reason to add xunit support to nose-timer since it's already built-in with nose.

I'd recommend adding something to the documentation/readme about --with-xunit providing machine-readable timings.

(If I were generating a report for people to look at, I'd reconsider python-timer though! Nice work.)

skudriashev commented 10 years ago

So then, probably, this issue is irrelevant. Right?

aehlke commented 10 years ago

Yes, IMO close it and just add a pointer to --with-xunit for those who want to integrate timer output with other tools. Unless I'm missing something that this plugin does that the xunit plugin doesn't that is useful for this case.

mahmoudimus commented 10 years ago

@aehlke @skudriashev that's a fair point, my initial inspiration was how a service like coveralls.io just does test coverage and alerts on test coverage variange. I was wondering how to define a standard for test timings, which a key metric to track when enforcing testing on a team.

It could be that the answer is, "we're xunit compatible" -- until it is no longer a viable solution for us.

skudriashev commented 10 years ago

@mahmoudimus, we can do that :)

mahmoudimus commented 9 years ago

Closing in favor of just using --with-xunit. Will re-address until this is no longer a viable solution.