Fix FDR part to support in massive amount of formulas

metaspace2020 / Lithops-METASPACE

Lithops-based Serverless implementation of the METASPACE spatial metabolomics annotation pipeline

12 stars 4 forks source link

Closed omerb01 closed 4 years ago

omerb01 commented 5 years ago

main changes:

formula_to_id dictionary now created in the stage when we build the database.
formula_to_id created in chunks.
each process of build_ranking() (FDR part) now iterates over formula_to_id chunks to search for formula_is

I think that we can optimise build_ranking() a lot by reducing the number of actions we launch. this will be discussed later.

omerb01 commented 5 years ago

@LachlanStuart ready for a review