BioplatformsAustralia / bpaotu

OTU database access for the Australian Microbiome
GNU Affero General Public License v3.0
5 stars 1 forks source link

download status switching to "not available" #238

Closed abissett closed 2 years ago

abissett commented 2 years ago

on bpa-otu staging, Under the metagenome tab, when a sample is selected (select metagenome, hit search to populate the sample list at the bottom of the page, select sample (e.g., 141251), usually, initially the expected page appears, with links to the data. If this page is left up, after some time the links (often) spontaneously disappear and are replaced by text saying “not available”. Occasionally this “not available” response is met initially (the data links are never populated).

hou098 commented 2 years ago

Looks like a performance problem fetching the trait options. It could be a database or caching problem on bioplatforms.com, or perhaps the trait-options search code in bpaotu needs some optimisation.

What's happening is that the metagenome search interface issues a request for the trait options when it appears. In the metagenome case on staging, that request is taking minutes to complete. Normally it takes a fraction of a second to populate the traits dropdown menu.

In the meantime, you've done a metagenome search and popped up the download dialog.

When the trait options request finally returns, (sometimes minutes later) it's causing the app to alter its state in such a way that clears its list of what's available to download. It's basically not expecting the request to return after such a massive delay.

This is technically an application bug, but "fixing" it to grey-out the search button for a few minutes while the traits options load after a clearly unacceptable delay wouldn't be a viable fix. I suspect it is only apparent due to the terrible database or cache performance on staging.bioplatforms.com

You can observe this by opening up the "inspector" in firefox or chrome and switching to the "network" tab. Requests for https://staging.bioplatforms.com/bpa/otu/private/api/v1/trait-options?amplicon={"value":7,"operator":"is"} take minutes to complete. This also happens on the plain search interface when switching the amplicon to, say, ITS1FITS4_fungi_ITS1 or metaxa_from_metagenome.

On my development box, the request for http://localhost:3000/private/api/v1/trait-options?amplicon={"value":7,"operator":"is"} in the metagenome case is pretty quick: 193ms.

Worryingly, https://data.bioplatforms.com/bpa/otu exhibits the same behaviour for some of the amplicon selections too. e.g. 1391fEukBr_v9_eukaryote

GET | https://data.bioplatforms.com/bpa/otu/private/api/v1/trait-options?amplicon={"value":1,"operator":"is"}
Waiting: 1.69 min

Those requests are supposed to be cached, but the slow ones are always slow, which makes me think there may be a redis cache problem on bioplatforms.com or a bug in the caching logic in bpaotu.