Open RubenVerborgh opened 1 month ago
In comunica we use http://w3c.github.io/rdf-tests/sparql/sparql11/manifest-all.ttl, could you try that one?
I'm guessing some breaking changes were introduced in http://w3c.github.io/rdf-tests/sparql/sparql11/manifest.ttl for the RDF 1.2 tests.
Thanks! That's indeed quite confusing, especially given that they broke URIs repeatedly (https://github.com/RubenVerborgh/SPARQL.js/commit/0ef0d4c3b9c44f1c9e075f8ac6837b34bf728466).
With npx rdf-test-suite spec/parser.js http://w3c.github.io/rdf-tests/sparql/sparql11/manifest-all.ttl -s http://www.w3.org/TR/sparql11-query/ -c .rdf-test-suite-cache/
, I get: ✔ 301 / 301 tests succeeded! (skipped 198)
. The 198 still seems like a lot, but perhaps that's okay?
For npx rdf-test-suite spec/parser.js http://w3c.github.io/rdf-tests/sparql/sparql11/manifest-sparql11-update.ttl -s http://www.w3.org/TR/sparql11-update/ -c .rdf-test-suite-cache/
, I unfortunately still have ✔ 0 / 0 tests succeeded!
.
Suggestion: still error on 0/0
?
I get: ✔ 301 / 301 tests succeeded! (skipped 198). The 198 still seems like a lot, but perhaps that's okay?
That should be fine indeed, since the passed runner is configured to skip all non-parsing tests: https://github.com/RubenVerborgh/SPARQL.js/blob/main/spec/parser.js#L4-L15
For npx rdf-test-suite spec/parser.js http://w3c.github.io/rdf-tests/sparql/sparql11/manifest-sparql11-update.ttl -s http://www.w3.org/TR/sparql11-update/ -c .rdf-test-suite-cache/, I unfortunately still have ✔ 0 / 0 tests succeeded!.
http://w3c.github.io/rdf-tests/sparql/sparql11/manifest-all.ttl should also work here. At least it does in comunica: https://github.com/comunica/comunica/blob/master/engines/query-sparql/package.json#L54-L56
Thanks, LGTM. Maybe just consider if you want the suite to error if it finds 0 tests.
Maybe just consider if you want the suite to error if it finds 0 tests.
Good idea, we should add that.
In addition, we may also want to add a flag that does not error in such cases, as there may be cases where zero tests are valid.
In SPARQL.js, running
npm run spec-base-query
now silently fails with ”0/0 tests”: