Open Jakob37 opened 3 weeks ago
northwestwitch
commented
3 weeks ago Hi Jakob, thanks for the explanation! The image is a bit tiny and doesn't show if I try to open it in a new tab. But looks like you are measuring the time it takes to load the single components of case page? What's the slowest file to load (I think the one one on top), the case page itself?
dnil
commented
3 weeks ago Hi there! Sounds great with some extra hands - there is plenty fun to do on many fronts.
I checked a few pages as well now, and note faster times than that, with 16k cases, but quite agree that the case page currently stands out as being sluggish. And agree with your image that the long wait is the first one in the flask view/controller before the main html document outline is in place. Let's start with profiling a bit in detail on the different case page component. I suspect we still could use a few projections for the variantS and panelS that are touched, and as you that some computation regarding eg panels need not be updated quite as often as it is now - by e.g. keeping track of the latest gene and panel changes.
dnil
commented
3 weeks ago On a sidenote you also need to update your certificates - your connections to your server show as unprotected. ;)
Jakob37
commented
3 weeks ago The image is a bit tiny and doesn't show if I try to open it in a new tab. But looks like you are measuring the time it takes to load the single components of case page? What's the slowest file to load (I think the one one on top), the case page itself?
Aha I see. I can open it, but see "private-user-images" in the URL. The main page load takes 6045 ms. The only other significant load time is from the images which takes in total ~1000 ms to load.
I checked a few pages as well now, and note faster times than that, with 16k cases, but quite agree that the case page currently stands out as being sluggish. And agree with your image that the long wait is the first one in the flask view/controller before the main html document outline is in place. Let's start with profiling a bit in detail on the different case page component. I suspect we still could use a few projections for the variantS and panelS that are touched, and as you that some computation regarding eg panels need not be updated quite as often as it is now - by e.g. keeping track of the latest gene and panel changes.
OK yes, that sounds good!
On a sidenot you also need to update your certificates - your connections to your server show as unprotected. ;)
🫣 Yes, well spotted. We are working right now (far too late) at transitioning to https!
northwestwitch
commented
3 weeks ago Functions to speed up:
[ ] Matching causatives. Takes most of the time for Lund instance
[x] Default panels. Takes quite some time and it's the major problem for us
[ ] Safe genes filter Takes quite some time in Lund. --> This has not been fixed entirely, see closed PR #4658. But we have merged a fix that avoids duplicates in the list of the safe genes and it should improve the situation in the case of many panels with repeated genes.
Is your feature request related to a problem in the current program to new available techology or software? Please describe and add links/citations if appropriate.
We would like to speed up interactions with the case page, such as adding comments and assigning yourself to the case.
Describe the solution you'd like
Possible approaches to speeding up subsequent interactions with the case page:
Describe alternatives you've considered
The ideal would be to reduce the initial page load time at a Mongo level or initial case load (for instance by having slow calculations precalculated). But suspecting this is hard nut, as this has been on the table for quite some time. Let me know if you have a different view.
Additional context
The loading time for certain pages such as the case page is perhaps the most pressing issues for our geneticists, and they constantly mention it. When working in their biggest institute (~5000 cases) a single page interaction does on a good day take ~7 seconds.
We recently tried moving out our mongodb to run "directly on the metal" instead of inside a VM + container. This did not do a huge difference to the case load for us for the case load. So I think we will need to address this somehow.
Here is a case load on a good day (heavily censored, wasn't sure what might be sensitive 😄 )
Let us know what you think! I think we in Lund could be able to spend quite some time on helping out exploring this, as it is of such pressing need for us.