Open rpompaideasoft opened 5 years ago
Regarding query performance:
(please share some examples when possible)
Did you tune any memory settings? please verify with MOnex the index usage (though I don't expect a large performance change compared to 4.5)
Regarding query performance:
* what kind of queries ?
access multiple files in multiple collections many times
did you define indexes? yes, on many collections
what kind of XML structures Nested XML Many nodes inside
how many XML documents, what is the typical size 32000 XML files are read in this query, each one between 200bytes and 3Kb of size The generated file exceeds one million lines and 30 Mb in size
Index Usage: existdb 4.5.0:
existdb 4.7.0:
Did you tune any memory settings? This is the config file: conf.zip
@rpompaideasoft we still don't know what your query does and how it is written. In order to help you we need a minimal reproducible example.
This includes
access the files
minus all unnecessary partscollection.xconf
(multiple?) so we see what indexes are defined, The fastest way to get us debugging is combining all of the above in an xqsuite test, but a link to a small xar would also work.
@rpompaideasoft we still don't know what your query does and how it is written. In order to help you we need a minimal reproducible example.
This includes
* an xquery that shows us how you `access the files` minus all unnecessary parts
Query with lucene: let $idPhraseQuery :=
, $result := fn:collection($col-path)//custom-ns:id[ft:query ( . , $idPhraseQuery)]/.. {$id} Index usage: existdb 4.5.0 Calls: 9371 Timing: 44.417 existdb 4.7.0 Calls: 9371 Timing: 176.425
Files: 1300 xml files
collection.xconf:
@rpompaideasoft there have been quite a few changes with respect to preformance. Can you test with 4.7.1
and the HEAD
of develop
? Chances are the performance improvements will be a 5.x feature. See https://github.com/eXist-db/exist/pull/2962
What is the problem
Using existdb 2.2 and 4.5 the quries take 10 minutes Using existdb 4.7 the same queries take 20 minutes
What did you expect
Questions
Context information