search

icaruseu / mom-ca

Monasterium.net (http://www.monasterium.net/mom) - repository and collaborative archive
https://github.com/icaruseu/mom-ca/wiki
GNU General Public License v3.0
17 stars 11 forks source link

Load user collection overview can take very long #1034

Open GVogeler opened 3 years ago

GVogeler commented 3 years ago

The user collection https://www.monasterium.net/mom/IlluminierteUrkundenFrankreich/my-collection currently takes several minutes to load, which can even result in a timeout.

GVogeler commented 3 years ago

Working assumption to avoid this is that the collection includes a huge number of hooked in charters, so calculating the temporal distribution for the navigation is very expensive (taking each single entry in the collection -> finding corresponding entry in the public collections -> extract date -> insert the date into the output -> sort the output -> break the output by chunk).

NTsch commented 3 years ago

Various tests with the Monex tool so far seem to indicate that the issue is related to the charter:public-entry function, which takes ~0.02 seconds to complete for one charter, which turns into minutes with thousands of charters.

StephanMa commented 3 years ago

Well... for me, it’s a really good value. 20ms is almost nothing for computing xml data in a Java based database. What you need is loading charter async... but unfortunately for this you have to rework the entire section (xquery and js parts)

StephanMa commented 3 years ago

Maybe related https://github.com/eXist-db/exist/issues/3918

GVogeler commented 3 years ago

For my understanding, the process, we have to accelerate, includes three steps:

  1. MOM-CA has to aggregate charters from different collections (where a single 20ms request via charter:public-entry() is not problematic, but with more than 2000 cases it sums up)
  2. sort them by date and
  3. build chunks for each 30 charters

How can we make the single steps more performant? And how can we separate reaction on the page from the time consuming functions (=include more async functionalities in the request)?

NTsch commented 3 years ago

The public version of the collection does not have the same issue. As far as I can tell, the reason is that it uses charters:years instead of mycharters:entries, and the latter is where the "slow" charter:public-entry() is called from.