search

w3c / rch-wg-charter

Charter proposal for an “RDF Dataset Canonicalization and Hash Working Group”
https://w3c.github.io/rch-wg-charter/
Other
12 stars 7 forks source link

Use-case idea #8

Closed pchampin closed 3 years ago

pchampin commented 3 years ago

Hi all, currently working on the RDF-star test-suite, I came to realize that updating the test-suite may cause problems. Suppose I detect a bug in a test, and I update the manifest accordingly.

Somebody loading an old EARL report and the new test-suite manifest might wrongly believe that the implementation described in the EARL report passes the test, while in fact in passed the earlier, buggy, version of the test. One solution would be for the EARL report to include a hash of the manifest it ran on.

Of course, for this solution to be complete, the manifest should itself include a hash for every file it points to, otherwise changing the files could pass undetected... But we don't have to figure out all the details right now.

What do you think of this use-case?

iherman commented 3 years ago

Sounds good to me. Now we have to be able to describe in some succinct form that even people who do not know what EARL is would understand...

msporny commented 3 years ago

Yes @pchampin, that's a valid use case... related:

https://www.w3.org/TR/vc-imp-guide/#referencing-other-credentials

gkellogg commented 3 years ago

In the particular case of an EARL report we could probably use a simpler mechanism where the commit that last modified the manifest(s) could be used both in the rollup implementation report.

Note that the EARL vocabulary doesn't have any notion of test manifest (only individual test URIs) and we'd need some vocabulary extensions to described manifests and their hashes, and there may be multiple manifests involved with a given report (e.g., SPARQL and JSON-LD). I typically add an earl:Report class and earl:assertion properties that serve as useful connective elements, and the Report might reference a hash of the manifests that were used, along with the URIs of those manifests, but if the hash isn't part of the manifest itself, it might be a high burden for implementors to calculate this hash.

As the implementation report incorporates elements of the manifests and the individual EARL reports submitted by implementors, it may be that an individual report should be ignored if the timestamp of the submission file is older than any of the manifests which include tests that it reports on.

It might be a reasonable time for a CG to give EARL an update.

iherman commented 3 years ago

@pchampin do you think you can put in a use case to the explainer? Having such use cases is important...

I would prefer to close this issue once a PR is there. Let us not give the impression that there are still open issues re charter...

pchampin commented 3 years ago

@gkellogg Thx for these precisions on EARL. Indeed, this would need to be thought through in more detailed. +1 to give EARL an update :) @iherman I created PR #58 which adds a new UC, slightly more general than the one suggesed above. I will therefore close this issue, the discussion can now happen on in #58.