Implementation reports

[csarven]

Where applicable, specifications should link to a document about the implementation reports - a summary/index. In addition to making implementations discoverable via specification, it communicates what could be used with their coverage. It provides supporting data/evidence for success/exit criteria ( https://github.com/solid/specification/issues/373 ) as well as the conformance criteria ( https://github.com/solid/specification/issues/282 ).

The summary/index (in RDF) can be similar to https://www.w3.org/TR/ldn/ linking to https://linkedresearch.org/ldn/tests/summary where individual implementation report (in RDF) can also be found. Where the projects implementing the specifications are also described in RDF... we end up with this (overview): https://csarven.ca/media/images/articles/linked-specifications-reports.svg .

This will generally require the summary/index document to be either updated manually or generated automatically, e.g., based on reports submitted (typically by the maintainer of the project implementing the specification) to a repository or to an inbox - the latter is the approach taken with LDN /ldn/tests/reports/ - and conforming reports are included in the summary.

For the summary and individual implementation reports, we'll need to allocate a URI space (similar to publishing technical reports /TR/{id})

[michielbdejong]

Great that you bring this up! This is the work of the test suite panel, of which you are a member, and the reports are currently at https://github.com/solid/test-suite#table

We are planning to move it to https://github.com/solid-test-suite but we can also move it elsewhere. :)

We should also link to this from the website, https://solidproject.org .

Whenever the contents of the table changes, we will inform the community through the W3C-CG mailing list.

[michielbdejong]

And just a side note (we can discuss this further within the test suite panel), although the specification-tests and the harness that runs them are both making good progress, there are still a lot of things that they don't cover. For instance websockets (see https://gitter.im/solid/test-suite?at=623300839a09ab24f3aae200 ). So for those we are still using the jest-based tests. Eventually the goal is to move everything over, but as you know the tests-in-RDF project was started over 3 years ago, so just to manage everybody's expectations, you should expect this to probably take another year or two before that move it complete.