Establish a workflow so we get updated ShEx/SHACL representations from Schema.org

google / schemarama

Schemarama is a project exploring standards-based validation for structured data, especially Schema.org.

Apache License 2.0

133 stars 23 forks source link

Establish a workflow so we get updated ShEx/SHACL representations from Schema.org #52

Open danbri opened 2 years ago

danbri commented 2 years ago

Goals

We should keep up to date with schema.org, and automate as much as possible.
Other schemas and projects should also be able to feed into this.
For these newer uses, we should use the https: URIs for Schema.org terms (prefix 'sdo:' ideally)

Background

In Schema.org repo, there is a utility shex_shacl_shapes_exporter.py
When run, it asks for ".nt terms definitions path". Use this: data/releases/14.0/schemaorg-all-https.nt
recent output:
- -rw-r--r-- 1 danbri eng 341956 Sep 14 13:53 ../../res.shacl.
- -rw-r--r-- 1 danbri eng 937836 Sep 14 13:53 ../../res.shexj
- these are available in demo/validator/sh/.sh*

danbri commented 2 years ago

/cc @RichardWallis @ericprud

I believe we should likely be using Github Actions, either pulling/polling from here, or pushing out from Schema.org upon release. I haven't fully surveyed the patterns, options and issues but this blog post looks like a good start.

danbri commented 2 years ago

RTFM'd a little more. It seems we could install an action in schemaorg/schemaorg repo that uses this: https://github.com/peter-evans/repository-dispatch ... and which would be wired to an action here

RichardWallis commented 2 years ago

Following the merge of schemaorg PR #3182 the output from the shex_shacl_shapes_exporter.py script will be produced when the site is built. On the release of a schema version (next one being V15.0) those files will be copied to the schemaorg/data/releases/{version number} directory in the repo. These could then be pushed, to or pulled from, another repo such as this via an action.

Would be good if the action could work with the 'latest' schema version.

As per the schemaorg PR comments, we should work on the naming of these files.

danbri commented 2 years ago

suggest 'shapes' instead of 'res'

RichardWallis commented 2 years ago

Schemaorg 6fcbc1b implements @danbri's suggestion.

danbri commented 2 years ago

Fab, thanks @RichardWallis

can we test run it on webschemas before the next real release?

RichardWallis commented 2 years ago

Provided./software/util/buildsite.py --release is run before deployment to weschemas the download should work from https://webschemas.org/version/latest/schemaorg-shapes.shexj

danbri commented 2 years ago

Aye aye cap’n!