ke4
commented
5 months ago It looks like that the root cause is that we are storing the same huge data in 2 different classes:
- ExperimentDesign class
- Experiment class (in its various subtypes)
We create the ExperimentDesign class with a huge structured data set first. That class is not very well designed. It can be split into more than one classes.
After create the ExperimentDesign class we create the Experiment class and copy some huge data structures from the to the Experiment class. So we have them twice now in memory.
We merged some PR to atlas-web-core and atlas-web-single-cell repositories related to Experiment Design refactor.
It looks like that these changes using even more memory as we can't start the web app with them on
stagingwhere we have lots of experiment. The web app gets memory issues when it tries to load a huge experiment and it stops there. The original goal of this change was to use less memory in the web app. With just a couple of experiments in the environment, like in our local environment it is working fine.We have to try to identify the root cause of this issue and create a fix for that and test it in
testandstagingenvironment.Steps:
I already created a new ticket to cover this issue: Adding experiment design as a field to Experiment entity cause a memory issue. I am going to reuse this existing ticket to reinvestigate the work Haider did and identify the part that is good for possible reuse/refactor for the next phase for experiment design refactor: putting experiment design into DB and read the data from there to display it in the web app.
Identify the reusable code in: