grambank / pygrambank

Apache License 2.0
4 stars 1 forks source link

speeding up sourcelookup? #70

Closed HedvigS closed 2 years ago

HedvigS commented 2 years ago

I suspect that the function sourcelookup in pygrambank is slowed down by going through all of hh.bib and gb.bib.

Is there a function we can have that loads these two in a simple format in the python environment so that it doesn't need to take so long every time?

xrotwang commented 2 years ago

Caching such things introduces

Also, code doing caching is really difficult to test.

So, I'd rather not do that, because I think it's quite a task if done properly.

HedvigS commented 2 years ago

Okay.

In that case, could sourcelookup please accept more than one sheet at a time so I can set it to report on a set of sheets and go do something else and then check back after a good while?

HedvigS commented 2 years ago

I'd like this to work please

grambank --repos /Users/skirgard/Git/Glottobank/Grambank sourcelookup /Users/skirgard/Git/glottolog/glottolog "original_sheets/JE_alun1238.tsv, original_sheets/JE_ford1242.tsv"

HedvigS commented 2 years ago

or that I can give it a txt files with the file paths on a new row, that'd be even sweeter.