Set up some standardised benchmark tests

[SimonGreenhill]

We should have some benchmarks to check performance against -- especially a good size LexStat analysis or something.

Alternatively since nose is being used for testing, could time all tests using nose-timer?

[xrotwang]

I think just a function formulated in LingPy 2.5 (including the input data) would be nice for me. This would give me an idea of

• what functionality would be a minimal viable product
• what performance is expected for this functionality

This function can just be handed to timeit.timeit, and the benchmark is ready to go.

[LinguList]

Okay, you are thinking of the general methods for cognate detection here? Maybe also alignments? I'll try and submit something later today or early tomorrow.

[SimonGreenhill]

I meant something like this:

import cProfile

cProfile.run("""
from lingpy.basic.workflow import Workflow
wf = Workflow('KSL.qlc')
wf.cognate_detection(export='tsv,html')
""")

(which doesn't really tell me much, to be fair, although there seems to be a LOT of calls to len(..) = 130k? )

@xrotwang I guess the workflow example is a good candidate for a m.v.p.?

[LinguList]

As to the workflow-class, I'd say we discard it, as workflows are to complicated to handle the possibilities, and we can put them in howtos, or do you think they are important?

As to the profiling: good point. If we want to get a bit more speed, it seems like the right time to think about it, and it is possible, that I introduced some stupid things (I was young and foolish...)

[SimonGreenhill]

No, I agree about workflows being removed in favour of howtos. And premature optimisation is a bad thing, but it'd be worth keeping an (not very close) eye on given the rewrite means you can change API etc.

[LinguList]

Yep, and for me it is also important that, say, things which are more or less optimized or work fast enough, should not be much slower than lingpy2.