UUDigitalHumanitieslab / EDPOP

A virtual research environment (VRE) that lets you collect, align and annotate bibliographical and biographical records from several online catalogs.
BSD 3-Clause "New" or "Revised" License
1 stars 0 forks source link

Pruning records is very slow #166

Closed tijmenbaarda closed 2 months ago

tijmenbaarda commented 2 months ago

In the method _get_content_graph() of SearchGraphBuilder the function remove_from_triplestore() is called. This function is very slow. Autocommit in rdflib is already off.

For interface testing, we can for now still disable saving (and removing) from the triplestore altogether, but we will need a solution for this.

This can be reproduced using the following curl command:

curl -X GET http://127.0.0.1:8000/api/catalogs/search/\?catalog=https://edpop.hum.uu.nl/readers/kb\&query=boek

The first time this is still quite fast, but after that it takes several seconds.