Closed jeswr closed 11 months ago
gkellogg
commented
1 year ago I could see a manifest for all RDF tests and another for SPARQL (which pretty much already exists).
We have to decide on how to handle experimental or newly proposed tests; additional manifests is one way to go, another would be to add some form of an aspect to each test entry. JSON-LD took the unified manifest approach and added options which could include specVersion and some other properties.
jeswr
commented
1 year ago Personally I would be in favor of having something similar to rdft:Approval that is applied to entire test suites or manifests rather than individual tests. The possible status' that I would be in favour of are
rdft:WGApprovedTestSuiterdft:WGProposedTestSuiterdft:CGApprovedTestSuiterdft:CGProposedTestSuiterdft:RejectedTestSuite (not likely to be used?)
With it being an open question as to whether rdft:WGApprovedTestSuite is a subclass of rdft:CGApprovedTestSuite.
Some N3 inference rules to determine whether each individual test is approved or not would be a great bonus :)
afs
commented
1 year ago Tests are likely to be developed within a working or community group repo for IP & copyright reasons and github repo access reasons.
afs
commented
1 year ago There is a SPARQL one for "current" - /sparql/sparql11/manifest.ttl.
This is the SPARQL 1.0 and SPARQL 1.1 test suites, brought up to RDF 1.1. It covers query, update and results.
"all" is tricky. If a new set of features appears (e.g. RDF-star), then a system will need to have an implementation so picking up "latest" during CI/CD isn't automatically the right thing to have. So the first question is what is the purpose - to test the boundaries or establish a solid core set of data exchange.
A top level manifest for the core set of languages (a general expectation of any system) would be useful.
Any system can mange their own set - this is the web!
BTW some test suites are un-passable - they contain alternatives. Tests are a communication tool for WGs and CGs.
Would it be possible to have a universal rdf-tests manifest that points to the manifest of each test-suite as a sub-manifest. That way tools like https://github.com/rubensworks/rdf-test-suite.js/ would be discover relevant test suites based on that single manifest rather than each tool using it having to specify the manifests urls to use for each tests case.
Ideally there would also be a universal manifest for all test-suites still in flux (e.g. RDF-Star and the renewed N3 effort https://w3c.github.io/N3/spec/).