Biom optimisation - Githubissues

BioplatformsAustralia / bpaotu

OTU database access for the Australian Microbiome

GNU Affero General Public License v3.0

5 stars 1 forks source link

Biom optimisation #43

Closed grahame closed 6 years ago

grahame commented 6 years ago

Speed up export:

a bug in the query for the relevant OTUs was doing an unrestricted cross-join between OTUs and Samples, which lead to a huge number of rows
we were using .index rather than a dictionary for lookups in arrays with thousands of elements
we were using .format in a busy loop, quite a large speedup by swapping to string primitives

Overall this code is now about 15 times faster