YAML-LD UCRs

[VladimirAlexiev]

Related to https://github.com/w3c/json-ld-syntax/issues/389, https://github.com/json-ld/yaml-ld/issues/3. TODO: scan these issues to extract more requirements.

Like any great W3C work, we should start with a Use Case Requirements specification / collection.

Until we have a repo (@gkellogg) where we can collect UCR per issue, let's use the following format. It could help UCR editors collect the issues:

• [ ] PRIO: Title (@ contacts)
  • description
  • considerations
  • considerations

Notes:

• The checkbox shows which UCRs have been posted as distinct issues in this repo
• PRIO is MUST/COULD/SHOULD and is approximate and subjective, so let's not spend too much effort discussing them. This is specifically not a "requirement compliance" rating
• Can include contacts of people who suggested the idea, or could be interested to develop it further. Not intended to track authorship precisely
• Give a detailed description
• Give as many relevant considerations, examples and links as you can

Here we go

---

• [ ] MUST: Least Surprise (@pchampin)

  • YAML-LD should work the same as JSON-LD unless specific processor options are selected
  • Since YAML is an extension of JSON, YAML-LD->RDF should produce the same as YAML->JSON-LD -> RDF,
  • YAML-LD should cover all features of JSON-LD 1.1

• [ ] MUST: Human Readability (@anatoly-scherbakov)

  • One of the most important reasons to use YAML-LD is YAML's superior human readability and writability compared to JSON
  • YAML-LD should provide "syntactic sugar" options to reduce the need for quoting (eg $ instead of @)
  • Eg see https://metacpan.org/pod/RDF::TrineX::Parser::Pretdsl (@tobyink)

• [ ] MUST: Compatibility (@gkellogg)

  • YAML-LD must be compatible with other related syntaxes (see "Polyglot Modeling")
  • Eg using the $ sigil ("namespace") will overlap with other existing uses. For example, JSON Schema has $schema, $vocabulary and other other keywords that would not overlap with the JSON-LD keyword namespace (that uses the @ sigil)

• [ ] SHOULD: YAML Intro (@VladimirAlexiev)

  • The YAML-LD spec or UCR spec should provide an introduction to YAML, since the YAML spec is rather technical
  • In particular, it should cover YAML extensions compared to JSON

• [ ] SHOULD: Archetypical Examples (@VladimirAlexiev)

  • The UCR spec should include archetypical YAML examples from various domains, eg software projects, modules, issues, etc

• [ ] SHOULD: Shortcuts (@VladimirAlexiev)

  • Should include some shorthand notations (micro-DSLs) for various purposes, eg
  • https://metacpan.org/pod/RDF::Trine::Parser::ShorthandRDF and https://www.w3.org/wiki/ShorthandRDF (@tobyink)
  • https://metacpan.org/pod/RDF::TrineX::Parser::Pretdsl in particular for CPAN modules described using DOAP (@tobyink)
  • ← for rdfs:domain and → for rdfs:range (@anatoly-scherbakov)
  • Schema Salad concatenation of identifiers along the hierarchy of scopes (@tetron)

• [x] COULD: Versions (@VladimirAlexiev)

  • Support all relevant YAML versions. These could include https://yaml.org/spec/1.0, https://yaml.org/spec/1.1, https://yaml.org/spec/1.2.2/
  • But YAML is pretty stable: 12 years between 1.2.1 and 1.2.2 (https://yaml.com/blog/2021-10/new-yaml-spec/), so maybe just the latest one is relevant

• [ ] SHOULD: Extensions (@VladimirAlexiev)

  • Leverage YAML features over JSON. Quoting from the YAML spec "The YAML 1.0 specification was published in early 2004 The YAML 1.1 specification was published in 2005. Around this time, the developers became aware of JSON. By sheer coincidence, JSON was almost a complete subset of YAML (both syntactically and semantically). The YAML 1.2 specification was published in 2009. Its primary focus was making YAML a strict superset of JSON".
  • Some of them can be quite useful in an RDF context:
  • Anchors and Aliases can represent non-tree graph structures and should mesh with JSON-LD Frames

• [x] Tags are comparable to datatypes #17.

  • the YAML json schema and core schema handle string, boolean, integer, float (the latter allows things like -.inf and .nan).
  • https://yaml.org/type/ handles a wider set, in particular dates and datetimes. But please note these are considered deprecated in 1.2 and are being removed in 1.3 https://github.com/yaml/yaml-spec/issues/268#issuecomment-1039000203
  • Maybe we should define a YAML schema to handle more xsd datatypes?
  • It should aim to eliminate problems related to the limited and non-standardized set of JSON literals. Eg the JSON number 12345678901234567890.12345 is converted to RDF literal "12345678901234567168"^^xsd:integer (see jsonld playground)
  • And could even work as a replacement of @type, eg

    
    # short form using tags
    dc:date: !xsd!date 2022-05-18

instead of long form

dc:date: @type: xsd:date @value: 2022-05-18

- [x] COULD: Polyglot Modeling #19
  - For efficient RDF modeling, you need to define multiple related artefacts: ontology, shapes (SHACL (@holgerknublauch) or SHEX (@ericprud)), JSON-LD context, maybe JSON-LD frames, JSON schema or Avro schema
  - Many communities like LD expression of their data, but mostly care about defining it with JSON schema (eg `w3c-ccg` @OR13 @nissimsan @msporny)
  - Many people have expressed the desire to define a unified or "technology independent"
  - Most polyglot frameworks are YAML-based. Examples include:
    - [LinkML](https://linkml.io/) ([github](https://github.com/linkml)) (@cmungall)
    - [FHIR](https://www.hl7.org/fhir/) (@ericprud: this is not YAML-based, can you provide a link to implementation?)
    - [Schema Salad](https://www.commonwl.org/v1.2/SchemaSalad.html) (@mr-c @tetron)
    - [A.ML](https://a.ml/) and [cloudinformationmodel](https://cloudinformationmodel.org/about.html)
    - [SOML](https://platform.ontotext.com/3.0/soml/index.html) (@VladimirAlexiev)
  - See eg https://github.com/w3c-ccg/traceability-vocab/issues/296 for a brief list of modeling framework requirements
  - YAML-LD should not take over these modeling-framework efforts, but should show how they can be used together, show archetypical examples, and maybe make a comparison
- [x] #20
  - YAML-LD must include comprehensive conformance tests for all its features.
  - In fact the features should be designed after agreed archetypical examples, so it's feasible for the development process to be "test-driven"
  - It should replicate/re-render all JSON-LD tests. Despite no official support in the spec, the gazillion JSON-LD conformance tests are written in YAML: https://github.com/w3c/json-ld-syntax/tree/main/yaml 

- [x] SHOULD #42 
- [x] MUST #43

[ioggstream]

While working on +yaml media type definition for expressing json files (openapi+yaml, ld+yaml, ..) we provided some interoperability considerations in https://ietf-wg-httpapi.github.io/mediatypes/draft-ietf-httpapi-yaml-mediatypes.html

We are engaging with YAML community via https://app.element.io/#/room/#chat-mediatype:yaml.io and it could be useful to discuss specific points together.

[OR13]

We generate JSON-LD contexts from Open API Specifications in YAML here:

https://w3c-ccg.github.io/traceability-vocab/openapi/

For example:

• https://w3c-ccg.github.io/traceability-vocab/openapi/#get-/schemas/common/AgActivity.yml

We invented a syntax for describing JSON-LD terms in JSON-Schema:

$linkedData:
  term: AgActivity
  '@id': https://w3id.org/traceability#AgActivity
title: Agricultural Activity

This allowed us to define an RDF type and a JSON Schema type in the same file, which helps us keep semantics and security in sync.

[ioggstream]

We invented a syntax for describing JSON-LD terms in JSON-Schema:

OT: @OR13 we are working on similar stuff, but with a different syntax after some analysis since we need to be compatible with OAS3.0 ( x-jsonld-context and x-jsonld-type). Here's an editor that does the mapping https://ioggstream.github.io/swagger-editor/

[ericprud]

The semantics of ShEx is defined in terms of its JSON(-LD) representation (e.g. value sets). I've so far relied on js-yaml to have the one true mapping from JSON to YAML in lirbaries like ShapePath. We'd brought up the idea of a YAML representation (which would, of course, be named "ShExY"), but since the translation was so trivial, we never found conflicts between the ways we wrote or interpreted ShExY. That said, if JSON-LD defined this, there'd be no need.

[OR13]

Sounds like we have a few different things happening:

1. JSON-LD -> YAML (objects with a @context)
2. JSON-LD Shape (shex) -> YAML shape (shexy)
3. JSON Schemas in YAML -> JSON-LD Context

(I think 3 might be a bit more different than 1 and 2).

I will elaborate a bit more on the why "why".

Most of the JSON-LD contexts we've seen have gotten really, really large... that made them really hard to maintain / test.

We set out to be able to test each RDF Type independently, and using JSON Schema as the base, since folks are used to composing JSON Schemas to manage complexity.

As a side effect, this allowed us to compose JSON Schemas to manage context complexity.

I suppose there are other tools we could have used to do this, but we picked OAS and JSON Schema because of their appeal to "non linked data people"... we've observed there are a lot off folks that stumble when they first encounter JSON-LD, we thought it would be good to give them smaller building blocks that were more familiar to them.

[ioggstream]

I think that since YAML 1.2 flow serialization "seems" JSON, it is almost impossible to define an ld+yaml that does behave in a different way from what the general audie ance expects.

Probably the only extension points (which do not have a JSON counterpart) are:

• the data type (e.g. float inf e nan);
• YAML features that allow describing a cyclic graph (e.g. using named anchors and alias nodes)

In general, I think that interoperability should be a concern since many implementations prefer serializing everything to JSON to avoid managing cyclic graph processing issues et al at runtime.

[gkellogg]

Repo has be set up at https://github.com/json-ld/yaml-ld. If you would like to contribute, and are a member of the JSON-LD Community Group I can add you to the contributors team. Please create an issue (or respond to an already existing issue) to be added to the team.

[gkellogg]

Moving this issue to the yaml-ld repo.

[OR13]

Primary reason I am interested in YAML-LD is small, composable, human readable RDF definitions for the modern era.

Compatibility with JSON-LD is good, but I think we should be wary of giving YAML-LD the same amount of rope that JSON-LD supports, and we should make sure that all the benefits of YAML are available to RDF, not just the parts of YAML that translate directly to JSON.

[ericprud]

* [FHIR](https://www.hl7.org/fhir/) (@ericprud: this is not YAML-based, can you provide a link to implementation?)

@VladimirAlexiev , https://github.com/hapifhir/hapi-fhir is a prominent open-source Java server. As you osberved, there's no FHIR/YAML (though we make use of it in the build process).

I'll delete this comment once you ack it.

[VladimirAlexiev]

@ericprud acked but why delete? Cheers!

[VladimirAlexiev]

Closing this; I'll gradually move bullets from description to separate issues. If you've made important considerations in comments above, please post them as separate issues.

@OR13 and @cmungall, would you like to add something to #19?

[VladimirAlexiev]

@gkellogg Can we use this issue somehow to track the completeness of https://github.com/json-ld/yaml-ld/pull/37?

• can you edit the description? (I can even edit comments in this project, so you should be able to do to the same)
• currently the checkboxes show whether a separate issue is posted here. Maybe we need two checkboxes: "as issue" and "in spec"?
• @pchampin @anatoly-scherbakov @ioggstream don't you want to post some more UCR issues? There's some relevant text captured from you above.

[ioggstream]

@VladimirAlexiev I think as of now it would be useful to complete a first draft and start closing some issues. Without a document stub, other folks cannot understand what we are doing.

With the "profiling" idea we will be able to accomodate more and more use cases #35

[gkellogg]

@gkellogg Can we use this issue somehow to track the completeness of #37?

As this issue is closed, probably not the best place to track anything. If you want to use this use case for tracking milestones, then it should be re-opened. Or, we should look into GitHub Milestones.

• can you edit the description? (I can even edit comments in this project, so you should be able to do to the same)

PR #37 is just a point in time, more use cases will certainly be forthcoming, and we'll need to figure out how to reconcile conflicting use cases.

Presumably, the "Requirements" part of UCR should describe how the spec should address a given use case, of if the use case has been rejected.

[ioggstream]

Create two milestones: -00 and -future