search

quince-science / QuinCe

QuinCe is an online tool for processing and quality control of data from scientific instruments, with a primary focus on oceanic data.
https://quince.science
GNU General Public License v3.0
7 stars 8 forks source link

Second load of manual QC page is slower than the first one #2366

Open squaregoldfish opened 2 years ago

squaregoldfish commented 2 years ago

Initial tests suggest that there's something in the AuthenticatedFilter that spins its wheels for a while. Haven't checked any further.

Applies to all subsequent visits after the first one, so may be something to do with cleaning up data?

Profiling is showing some places where things can be improved, so follow those through. Also add progress bars to the loading dialog.

Profiling for:

MySQL queries/indexes for

squaregoldfish commented 2 years ago

This is due to garbage collection. The QC pages load a huge amount of stuff, and clearing it up later is horribly slow.

Try using the ZGC garbage collector (JDK 11+ on Linux, 14+ on Mac)

Also using a non-monolithic strategy (#2218, #2334) might help. QC page GC.log.zip

squaregoldfish commented 10 months ago

The most common reloading is after QC has finished, then recalculation, then reloading the QC page. Can we cache all this? See also #2675

squaregoldfish commented 5 months ago

https://docs.oracle.com/en/java/javase/17/gctuning/introduction-garbage-collection-tuning.html

squaregoldfish commented 5 months ago

Profiling is showing some places where things can be improved, so follow those through. Also add progress bars to the loading dialog.

Profiling for: