Open coret opened 1 week ago
These are different issues combined.
- query: file://generator.rq
Can you share your generator query? Even better, push your config to the configurations repository and link to the dump file that you’re using.
But the LD Workbench generates a SPARQL query which Qlever can't handle (yet) and I don't see how to change the LD Workbench behaviour.
LD Workbench does not generate this query, so it’s probably Comunica. Running QLever seems way too much work, so I’m not going to reproduce this locally. Compare this to a simple oxigraph start
.
Can you share your generator query? Even better, push your config to the configurations repository and link to the dump file that you’re using.
See https://www.github.com/netwerk-digitaal-erfgoed/ld-workbench-configuration/tree/main/nafotos-sparql-endpoint for config and https://nde-europeana.ams3.cdn.digitaloceanspaces.com/7-000spaondntfoto.2.zip for a big part of the NA photocollection (for the 7-000 stage).
Compare this to a simple oxigraph start.
Have not tries oxigraph yet, will do!
Have not tries oxigraph yet, will do!
Have not tried your code in PR 99 but tried to start and import the 2.8GB N-triple file directly:
$docker run --rm -v ./data:/data -p 7878:7878 oxigraph/oxigraph --location /data serve --bind 0.0.0.0:7878
$curl -f -X POST http://localhost:7878/store?default -H 'Content-Type:application/n-triples' --data-binary "@7-000spaondntfoto.3.nt"
curl: option --data-binary: out of memory
curl: try 'curl --help' or 'curl --manual' for more information
Hope your src/import.ts won't be bothered by the big filesize.
Have not tries oxigraph yet, will do!
Have not tried your code in PR 99 but tried to start and import the 2.8GB N-triple file directly:
$docker run --rm -v ./data:/data -p 7878:7878 oxigraph/oxigraph --location /data serve --bind 0.0.0.0:7878 $curl -f -X POST http://localhost:7878/store?default -H 'Content-Type:application/n-triples' --data-binary "@7-000spaondntfoto.3.nt" curl: option --data-binary: out of memory curl: try 'curl --help' or 'curl --manual' for more information
Hope your src/import.ts won't be bothered by the big filesize.
Use curl -T
instead to stream the file instead of loading the whole file in memory. The new import feature is streaming as well. Just not sure yet about the best YAML config conventions for it.
Your query returns no results, so I cannot test your pipeline. Please provide a ready-to-go reproducer.
I have loaded all NA photocollection N-triples (including the 3GB testfile "7") into my local (production) GraphDB and the LD Workbench works great.
The same iterator/generator doesn't work when I use the endpoint https://service.archief.nl/sparql:
The Generator did not run successfully, it could not get the results from : Invalid SPARQL endpoint response from https://service.archief.nl/sparql (HTTP status 400)
).I thought this would also be a good moment to test out Qlever. But the LD Workbench generates a SPARQL query which Qlever can't handle (yet) and I don't see how to change the LD Workbench behaviour.
I've also tested Fuseki (v5), but this (same generator/iterator as used in above cases) ends in an out-of-memory message in the LD Workbench. Adding a batchSize doesn't help.
part of my LD-Workbench configuration: