search

sympy / sympy_benchmarks

Some benchmarks of SymPy
13 stars 32 forks source link

Added a benchmark for cse of a long expression. #16

Closed moorepants closed 9 years ago

bjodah commented 9 years ago

+1

moorepants commented 9 years ago

@bjodah I see that you've been writing tests for some of the code. Do you care that I'm not?

I'm not sure I see the usefullness of the tests yet. Ideally we want benchmarks that are backwards compatible for a good portion of the history. This either requires sympy to be really good at being backwards compatible or for us to write crazy benchmarks that depend on versions of sympy. Ideally, we shouldn't have to test sympy functionality here because it is already tested in sympy at each commit that passes tests. What's the reasoning for adding the tests in this package?

bjodah commented 9 years ago

I had the idea that eventually I'd like only to include timings which actually gave the correct result (flagging the other timings as failed), but currently I don't think this is possible with asv, until then I just put them as ordinary tests. I tried to elaborate on my vision here: https://github.com/spacetelescope/asv/issues/296

So, no I don't think tests are mandatory for now, but in case we want them later on it will be easier to implement them if they are already in the repo...

moorepants commented 9 years ago

Ok, I'll merge this as is. Since I generate the inputs to my code with other software, I'll have to think about the tests for it.