ebi-gene-expression-group / atlas-web-single-cell

Single Cell Expression Atlas web application
Apache License 2.0
5 stars 5 forks source link

Revisit: load experiment design table from DB #409

Open ke4 opened 1 month ago

ke4 commented 1 month ago

We had an epic for rewriting the part of both Bulk and Single Cell Expression Atlas to read the data from the database and creating a table on the web page for experiment design. It suppose to replace the current design where we read this data from the experiment design file. When we merged these changes to the web apps the application could not load the data to the cache. After a brief debugging it turned out that the modified code is using too much memory when loading the data into the cache and eventually we get an OutOfMemoryError. These PRs were working in our local environment as we don't have lots of data locally, but they were failing in staging environment. Now we also have test environment, although we still need to ask Data Prod Team to load the same datasets to test that we have in staging to be able to do the same tests in both environments.

Here is the list of the reverted PRs that we can investigate.

In this ticket I collected the reusable changes/commits from the above PRs: Loading experiments into the cache is failing