search

kmcluskey / FlyMet

A multi-omics web app for Drosophila tissues
MIT License
3 stars 0 forks source link

Speed up Pathway page #56

Open kmcluskey opened 3 years ago

kmcluskey commented 3 years ago

The Pathway page is REALLY slow to load first time round - this can cause the server to time out.

joewandy commented 3 years ago

Is it slow because of this? https://github.com/kmcluskey/FlyMet/blob/master/met_explore/pathway_analysis.py#L194-L220

kmcluskey commented 3 years ago

I'll need to check - this is code from PiMP so not entirely sure - I'll check what parts are slow.

joewandy commented 3 years ago

Many codes that need to reconstruct dataframes from a large number of individual cells, stored as records in the database seem slow too? For example,

https://github.com/kmcluskey/FlyMet/blob/master/met_explore/compound_selection.py#L142-L159 

2020-10-30 13:15:34.888 | INFO     | met_explore.compound_selection:get_group_df:145 - Getting the peak group DF
2020-10-30 13:17:24.568 | INFO     | met_explore.compound_selection:get_group_df:165 - Generating the peak group dataframe took: 109.6802543 
2020-10-30 13:17:24.639 | INFO     | met_explore.views:peak_data:633 - returning the peak data
1:18

took nearly 2 minutes to run.

kmcluskey commented 3 years ago

A dump of the pals_df has now been added which helps with this but may not be ideal.