search

netwerk-digitaal-erfgoed / dataset-knowledge-graph

Pipeline that generates the NDE Dataset Knowledge Graph
European Union Public License 1.2
2 stars 0 forks source link

GraphDB imports take too long #52

Open ddeboer opened 9 months ago

ddeboer commented 9 months ago
Screenshot 2023-12-16 at 15 36 00
ddeboer commented 9 months ago

It seems large imports are blocking the rest, for example https://data.colonialcollections.nl/_api/datasets/nmvw/collection-archives/download.nt.gz (1.4 GB). We may want to cap imports at 1 GB or similar. If a distribution is larger, we simply cannot process it. We’ll have to recommend splitting the distribution into smaller parts.

ddeboer commented 8 months ago

Other GraphDB failures:

Screenshot 2024-01-16 at 13 32 56

We have disabled inferencing, so why we get this is beyond me.