Address json-ld process issues with `oneOf` schema optionality

[BenjaminMoe]

Description of Problem

oneOf is poorly supported by OAS / JSON Schema processors

For example, this is what renders in JSON Schema form when you use oneOf:

oneOf is not supported by the JSON-LD context builder

For example, this is the current context for Entity:

 "Entity": {
      "@id": "https://w3id.org/traceability#Entity",
      "@context": {}
    },

Proposed Approaches to the Problem

Approach 1: Remove oneOf schema selection with a fixed option: oneOf is not supported by our JSON-LD processor
Pull Request: https://github.com/w3c-ccg/traceability-vocab/pull/387
Current Conclusion: Rejected as their are too many use-cases where an Entity could be a Person or an Organization, and we can't assert that in all cases it will only be an organization.

Approach 2: Allow json-ld processor to break in a way to default to a sane default
Pull Request: https://github.com/w3c-ccg/traceability-vocab/pull/388
Current Conclusion: Rejected as this does not address the underlying JSON Schema processors and JSON-LD context builder technical issue.

Approach 3: Flatten schemas in a way that still allows for functionality without the use of oneOf
Pull Request: https://github.com/w3c-ccg/traceability-vocab/pull/396
Current Conclusion: This seems to be the most pragmatic approach for allowing optionality without implementing any changes to the JSON-LD processor. There are some opinions on the details, but there is consensus around this approach.

Approach 4: Update JSON-LD processor to support oneOf schema functionality Current Conclusion: No one has volunteered to implement this change

Approach 5: Add Enitity as a node, with subject be a ref Person or Organization

[OR13]

2 and 3 seem orthogonal to the main issue...

https://github.com/w3c-ccg/traceability-vocab/blob/4b17ce330bb6c804ef491d93e42c6a1f79584bec/docs/openapi/components/schemas/common/DCSAShippingInstruction.yml

example

Here is the problem:

<urn:epc:id:bic:YMLU3380910> <http://www.w3.org/1999/02/22-rdf-syntax-ns#type> <https://service.unece.org/trade/uncefact/vocabulary/uncefact/#TransportEquipment> .
<urn:epc:id:bic:YMLU3380910> <https://api.swaggerhub.com/domains/dcsaorg/DCSA_DOMAIN/1.0.1#/components/schemas/ISOEquipmentCode> "20G1" .
<urn:epc:id:bic:YMLU3380910> <https://service.unece.org/trade/uncefact/vocabulary/uncefact/#identificationId> "YMLU3380910" .
<urn:epc:id:bic:YMLU3380910> <https://w3id.org/traceability/#undefinedTermseals> _:c14n4 .
_:c14n0 <http://www.w3.org/1999/02/22-rdf-syntax-ns#type> <https://schema.org/PostalAddress> .
_:c14n0 <https://schema.org/addressCountry> "CN" .
_:c14n0 <https://schema.org/addressLocality> "Xindao" .
_:c14n0 <https://schema.org/addressRegion> "Fujian Province" .
_:c14n0 <https://schema.org/postalCode> "361100" .
_:c14n0 <https://schema.org/streetAddress> "Xin Fei Da Dao 139" .
_:c14n1 <http://www.w3.org/1999/02/22-rdf-syntax-ns#type> <https://w3id.org/traceability#CargoLineItem> .
_:c14n1 <https://api.swaggerhub.com/domains/dcsaorg/DCSA_DOMAIN/1.0.1#/components/schemas/descriptionOfGoods> "Rollators Cardboard Box" .
_:c14n1 <https://service.unece.org/trade/uncefact/vocabulary/uncefact/#physicalShippingMarks> "ByAcre Age is the new Black" .
_:c14n10 <http://www.w3.org/1999/02/22-rdf-syntax-ns#type> <https://w3id.org/traceability#CargoLineItem> .
_:c14n10 <https://api.swaggerhub.com/domains/dcsaorg/DCSA_DOMAIN/1.0.1#/components/schemas/descriptionOfGoods> "Rollators Organizer Bags" .
_:c14n10 <https://service.unece.org/trade/uncefact/vocabulary/uncefact/#physicalShippingMarks> "ByAcre Age is the new Black" .
_:c14n11 <http://www.w3.org/1999/02/22-rdf-syntax-ns#type> <https://w3id.org/traceability#CargoLineItem> .
_:c14n11 <https://api.swaggerhub.com/domains/dcsaorg/DCSA_DOMAIN/1.0.1#/components/schemas/descriptionOfGoods> "Rollators Cane Holder Bracket" .
_:c14n11 <https://service.unece.org/trade/uncefact/vocabulary/uncefact/#physicalShippingMarks> "ByAcre Age is the new Black" .
_:c14n12 <http://www.w3.org/1999/02/22-rdf-syntax-ns#type> <https://service.unece.org/trade/uncefact/vocabulary/uncefact/#Instructions> .
_:c14n12 <https://api.swaggerhub.com/domains/dcsaorg/DCSA_DOMAIN/1.0.1#/components/schemas/transportDocumentType> "MBL" .
_:c14n12 <https://service.unece.org/trade/uncefact/vocabulary/uncefact/#consigneeParty> _:c14n8 .
_:c14n12 <https://service.unece.org/trade/uncefact/vocabulary/uncefact/#consignorParty> _:c14n9 .
_:c14n12 <https://service.unece.org/trade/uncefact/vocabulary/uncefact/#freightForwarderParty> _:c14n3 .
_:c14n12 <https://service.unece.org/trade/uncefact/vocabulary/uncefact/#includedConsignmentItem> _:c14n7 .
_:c14n12 <https://service.unece.org/trade/uncefact/vocabulary/uncefact/#utilizedTransportEquipment> <urn:epc:id:bic:YMLU3380910> .
_:c14n12 <https://service.unece.org/trade/uncefact/vocabulary/uncl1153/#Consignment_identifier_carrier_assigned> "XMANHR2102045" .
_:c14n12 <https://w3id.org/traceability/#undefinedTermfirstNotify> _:c14n13 .
_:c14n13 <http://www.w3.org/1999/02/22-rdf-syntax-ns#type> <https://schema.org/Organization> .
_:c14n13 <https://schema.org/PostalAddress> _:c14n14 .
_:c14n13 <https://schema.org/description> "Better Lives Products" .
_:c14n13 <https://schema.org/email> "procurement@lifetech-example.org" .
_:c14n13 <https://schema.org/name> "Better Life Tech" .
_:c14n13 <https://schema.org/telephone> "+32-5555-8495" .
_:c14n14 <http://www.w3.org/1999/02/22-rdf-syntax-ns#type> <https://schema.org/PostalAddress> .
_:c14n14 <https://schema.org/addressCountry> "BE" .
_:c14n14 <https://schema.org/addressLocality> "Ramegnies-Chin" .
_:c14n14 <https://schema.org/addressRegion> "Hainaut" .
_:c14n14 <https://schema.org/postalCode> "7520" .
_:c14n14 <https://schema.org/streetAddress> "Rue de la Poste 272" .
_:c14n15 <http://www.w3.org/1999/02/22-rdf-syntax-ns#type> <https://schema.org/PostalAddress> .
_:c14n15 <https://schema.org/addressCountry> "BE" .
_:c14n15 <https://schema.org/addressLocality> "Ramegnies-Chin" .
_:c14n15 <https://schema.org/addressRegion> "Hainaut" .
_:c14n15 <https://schema.org/postalCode> "7520" .
_:c14n15 <https://schema.org/streetAddress> "Rue de la Poste 272" .
_:c14n2 <http://www.w3.org/1999/02/22-rdf-syntax-ns#type> <https://w3id.org/traceability#CargoLineItem> .
_:c14n2 <https://api.swaggerhub.com/domains/dcsaorg/DCSA_DOMAIN/1.0.1#/components/schemas/descriptionOfGoods> "Rollators" .
_:c14n2 <https://service.unece.org/trade/uncefact/vocabulary/uncefact/#physicalShippingMarks> "ByAcre Age is the new Black" .
_:c14n3 <http://www.w3.org/1999/02/22-rdf-syntax-ns#type> <https://schema.org/Organization> .
_:c14n3 <https://schema.org/PostalAddress> _:c14n6 .
_:c14n3 <https://schema.org/name> "Intertrans NV [378]" .
_:c14n3 <https://schema.org/telephone> "+32-3-201.98.10" .
_:c14n4 <http://www.w3.org/1999/02/22-rdf-syntax-ns#type> <https://service.unece.org/trade/uncefact/vocabulary/uncefact/#Seal> .
_:c14n4 <https://w3id.org/traceability/#undefinedTermsealNumber> "YMAI715692" .
_:c14n5 <http://www.w3.org/1999/02/22-rdf-syntax-ns#type> <https://w3id.org/traceability#CargoLineItem> .
_:c14n5 <https://api.swaggerhub.com/domains/dcsaorg/DCSA_DOMAIN/1.0.1#/components/schemas/descriptionOfGoods> "Rollators Backrest" .
_:c14n5 <https://service.unece.org/trade/uncefact/vocabulary/uncefact/#physicalShippingMarks> "ByAcre Age is the new Black" .
_:c14n6 <http://www.w3.org/1999/02/22-rdf-syntax-ns#type> <https://schema.org/PostalAddress> .
_:c14n6 <https://schema.org/addressCountry> "BE" .
_:c14n6 <https://schema.org/addressLocality> "Antwerpen" .
_:c14n6 <https://schema.org/postalCode> "BE-2100AN" .
_:c14n6 <https://schema.org/streetAddress> "Belcrownlaan 25 - 3rd floor" .
_:c14n7 <http://www.w3.org/1999/02/22-rdf-syntax-ns#type> <https://service.unece.org/trade/uncefact/vocabulary/uncefact/#ConsignmentItem> .
_:c14n7 <https://api.swaggerhub.com/domains/dcsaorg/DCSA_DOMAIN/1.0.1#/components/schemas/cargoLineItem> _:c14n1 .
_:c14n7 <https://api.swaggerhub.com/domains/dcsaorg/DCSA_DOMAIN/1.0.1#/components/schemas/cargoLineItem> _:c14n10 .
_:c14n7 <https://api.swaggerhub.com/domains/dcsaorg/DCSA_DOMAIN/1.0.1#/components/schemas/cargoLineItem> _:c14n11 .
_:c14n7 <https://api.swaggerhub.com/domains/dcsaorg/DCSA_DOMAIN/1.0.1#/components/schemas/cargoLineItem> _:c14n2 .
_:c14n7 <https://api.swaggerhub.com/domains/dcsaorg/DCSA_DOMAIN/1.0.1#/components/schemas/cargoLineItem> _:c14n5 .
_:c14n7 <https://schema.org/unitCode> "CBM" .
_:c14n7 <https://schema.org/unitCode> "KGM" .
_:c14n7 <https://service.unece.org/trade/uncefact/vocabulary/uncefact/#grossVolumeMeasure> "2.723E1"^^<http://www.w3.org/2001/XMLSchema#double> .
_:c14n7 <https://service.unece.org/trade/uncefact/vocabulary/uncefact/#netWeightMeasure> "1.6719E3"^^<http://www.w3.org/2001/XMLSchema#double> .
_:c14n7 <https://service.unece.org/trade/uncefact/vocabulary/uncefact/#packageQuantity> "245"^^<http://www.w3.org/2001/XMLSchema#integer> .
_:c14n8 <http://www.w3.org/1999/02/22-rdf-syntax-ns#type> <https://schema.org/Organization> .
_:c14n8 <https://schema.org/PostalAddress> _:c14n15 .
_:c14n8 <https://schema.org/description> "Better Lives Products" .
_:c14n8 <https://schema.org/email> "procurement@lifetech-example.org" .
_:c14n8 <https://schema.org/name> "Better Life Tech" .
_:c14n8 <https://schema.org/telephone> "+32-5555-8495" .
_:c14n9 <http://www.w3.org/1999/02/22-rdf-syntax-ns#type> <https://schema.org/Organization> .
_:c14n9 <https://schema.org/PostalAddress> _:c14n0 .
_:c14n9 <https://schema.org/description> "Advanced Production - Delivered" .
_:c14n9 <https://schema.org/email> "xxinau-sales@example.org" .
_:c14n9 <https://schema.org/name> "Xxinau Manufacturing Co. Ltd." .
_:c14n9 <https://schema.org/telephone> "+86-555-865-8495" .

[OR13]

In practice we don't actually use "Entity" for any useful semantics... because it doesn't actually define any:

https://github.com/w3c-ccg/traceability-vocab/blob/4b17ce330bb6c804ef491d93e42c6a1f79584bec/docs/openapi/components/schemas/common/Entity.yml

https://github.com/w3c-ccg/traceability-vocab/blob/gh-pages/contexts/traceability-v1.jsonld#L709

 "Entity": {
      "@id": "https://w3id.org/traceability#Entity",
      "@context": {}
    },

All of the semantics are coming from "Person" or "Organization"... meaning we don't actually use Entity for anything other than schema validation... which has nothing to do with JSON-LD at all... but logically, we don't really want to be doing that... there are cases where you would want to know that an Organization and not a Person was handling something...

Keeping oneOf in credentials means being ok with results like this:

{
    "type": "Person",
    "firstName": "Amelie"
}

^ If this is good enough for "Entity", I think we probably need to talk about what we are trying to do here.

[TallTed]

Re: "oneOf is not supported by our JSON-LD processor" (https://github.com/w3c-ccg/traceability-vocab/issues/395#issue-1216458166) -- "our" should be replaced by a comprehensible identifier of the entity/ies whose JSON-LD processor does not support oneOf. This is necessary because, for instance, neither I nor my employer have participated in joint production of a JSON-LD processor, so it is not "our" processor when I speak of it.

---

re "Here is the problem:" (https://github.com/w3c-ccg/traceability-vocab/issues/395#issuecomment-1110300096)

That is a bunch of triples. It is not a problem statement nor definition nor illustration. Please use plain language to state the problem which you apparently meant to illustrate by that dump of triples.

---

Re: "being ok with results like this" (https://github.com/w3c-ccg/traceability-vocab/issues/395#issuecomment-1110305879): when a Person is the accurate Entity type (i.e., the Entity is not an Organization), it does not make sense to require the attributes of an Organization. I fail to see why oneOf (Person, Organization) somehow makes the minimum required description of either the Person or Organization less than the minimum required description would be if the oneOf were dropped and either Person or Organization were always required.

It makes perfect sense to me to require some specific attributes of such an Entity; the specific list of which attributes are required would naturally vary with the subtype of that Entity, e.g., Person or Organization (there might be more relevant types that we haven't yet encountered in our far-from-exhaustive explorations), because, for instance, a dateOfBirth of a Person does not directly translate to dateOfIncorporation (or similar) for an Organization. Likewise, while firstName does not make sense for Organization and is insufficient (among other things) for a Person, legalName might make sense for either, and address might make sense for neither. (See https://github.com/kdeldycke/awesome-falsehood for more.)

---

Re: "I think we probably need to talk about what we are trying to do here" (https://github.com/w3c-ccg/traceability-vocab/issues/395#issuecomment-1110305879) -- Yes, I think we probably do.

[OR13]

Another alternative solution would be to remove any $ref that yielded a descendent that contained oneOf from all the certificate types we (Transmute) support.

This would not impact any of the other certificate types, and would resolve the broken behavior we are currently observing when handling the schemas in this repo, but only for "steel or e-commerce" certificates... which seems unhelpful to others.

See the proposal here for the best path forward that I am aware of currently: https://github.com/w3c-ccg/traceability-vocab/pull/396

[OR13]

@BenjaminMoe proposes essentially the same thing here, just with lots of optionality included...

https://github.com/w3c-ccg/traceability-vocab/pull/396#issuecomment-1111308829

and an ambiguous entityType attribute, with no defined semantics... however there is some precedent for that in schema.org, for example:

https://schema.org/additionalType

I prefer to not include so many optional fields...

[BenjaminMoe]

@OR13 I was thinking as I wrote that I shouldn't use the PR page for issue discussion.

My intended implementation of this would be merge Organization and Person into a new Entity.yml which would replace the existing oneOf Entity.yml. And then go back and re-generate all of the proofs.

title: Entity
description: A merge of Person and Organization
type: object
  type:
    oneOf:
      - type: array
        items:
          type: string
          enum:
            - Entity
      - type: string
        const: Entity
  entityType:
    description: The type of entity
    type: string
    enum:
      - Person
      - Organization
  name:
    title: Name
    description: Name of the organization.
    type: string
    $linkedData:
      term: name
      '@id': https://schema.org/name
  legalName:
    title: Legal Name
    description: The official name of the organization, i.e., the registered company name.
    type: string
    $linkedData:
      term: legalName
      '@id': https://schema.org/legalName
  firstName:
    title: First Name
    description: Person's first name.
    type: string
    $linkedData:
      term: firstName
      '@id': https://schema.org/givenName
  lastName:
    title: Last Name
    description: Person's last name.
    type: string
    $linkedData:
      term: lastName
      '@id': https://schema.org/familyName
  email:
    title: Person's Email Address
    description: Person's email address.
    type: string
    $linkedData:
      term: email
      '@id': https://schema.org/email
  phoneNumber:
    title: Phone Number
    description: Person's contact phone number.
    type: string
    $linkedData:
      term: phoneNumber
      '@id': https://schema.org/telephone
  worksFor:
    title: Works For
    description: Company which employs the person.
    $ref: ./Organization.yml
    $linkedData:
      term: worksFor
      '@id': https://schema.org/worksFor
  jobTitle:
    title: Job Title
    description: Person's job title.
    type: string
    $linkedData:
      term: jobTitle
      '@id': https://schema.org/jobTitle
  leiCode:
    title: Legal Entity Identifier Code
    description: Legal entity identifier code
    type: string
    $linkedData:
      term: leiCode
      '@id': https://schema.org/leiCode
  url:
    title: URL
    description: URL of the organization.
    type: string
    $linkedData:
      term: url
      '@id': https://schema.org/url
  description:
    title: Description
    description: Description of the company.
    type: string
    $linkedData:
      term: description
      '@id': https://schema.org/description
  globalLocationNumber:
    title: Global Location Number
    description: The GS1 GLN of the organization.
    type: string
    $linkedData:
      term: globalLocationNumber
      '@id': https://schema.org/globalLocationNumber
  address:
    title: Postal Address
    description: The postal address for an organization or place.
    $ref: ./PostalAddress.yml
    $linkedData:
      term: address
      '@id': https://schema.org/PostalAddress
  faxNumber:
    title: Phone Number
    description: Organization's fax number.
    type: string
    $linkedData:
      term: faxNumber
      '@id': https://schema.org/faxNumber
  brand:
    title: Brand
    description: Branding for this organization
    $ref: ./Brand.yml
    $linkedData:
      term: brand
      '@id': https://schema.org/Brand
  contactPoint:
    title: Contact Point
    description: Points of contact for this organization
    $ref: ./ContactPoint.yml
    $linkedData:
      term: contactPoint
      '@id': https://schema.org/ContactPoint
  taxId:
    title: Tax Identification Number
    description: >-
      The Tax / Fiscal ID of the organization or person, e.g., the TIN in the US
      or the CIF/NIF in Spain.
    type: string
    $linkedData:
      term: taxId
      '@id': https://schema.org/taxID
  iataCarrierCode:
    title: IATA Carrier Code
    description: Carrier's three-digit IATA airline code number
    type: string
    $linkedData:
      term: iataCarrierCode
      '@id': https://onerecord.iata.org/cargo/Company#airlineCode

The intended reason for including all of these optional fields is it means that no change would be needed for the JSON examples, only the proofs would need to be updated for everything that's already using Entity.

[OR13]

I don't think we want this many optional fields in Entity... @mprorock has indicated that there is likely a minimal set of required fields that would meet the regulatory guidelines... I would like them encoded in a schema that marked them as required... and to avoid copy pasting $ref: './Entity.yml' all over the place, because it probably does not yield the kind of data a verifier / regulator would want to receive.

ideally we would use more inline restricted schemas, and less $ref.

[BenjaminMoe]

Entity is used in Verifiable Business Card. I think we need the minimum requirements for defining an organization would be.

title: Entity
description: A merge of Person and Organization
type: object
  type:
    oneOf:
      - type: array
        items:
          type: string
          enum:
            - Entity
      - type: string
        const: Entity
  entityType:
    description: The type of entity
    type: string
    enum:
      - Person
      - Organization
  name:
    title: Name
    description: Name of the organization.
    type: string
    $linkedData:
      term: name
      '@id': https://schema.org/name
  legalName:
    title: Legal Name
    description: The official name of the organization, i.e., the registered company name.
    type: string
    $linkedData:
      term: legalName
      '@id': https://schema.org/legalName
  url:
    title: URL
    description: URL of the person or organization.
    type: string
    $linkedData:
      term: url
      '@id': https://schema.org/url
  taxId:
    title: Tax Identification Number
    description: >-
      The Tax / Fiscal ID of the organization or person, e.g., the TIN in the US
      or the CIF/NIF in Spain.
    type: string
    $linkedData:
      term: taxId
      '@id': https://schema.org/taxID

We could keep a small number of well-defined requirements, and then just say additionalProperties: true.

[mprorock]

Entity is used in Verifiable Business Card. I think we need the minimum requirements for defining an organization would be.

title: Entity
description: A merge of Person and Organization
type: object
  type:
    oneOf:
      - type: array
        items:
          type: string
          enum:
            - Entity
      - type: string
        const: Entity
  entityType:
    description: The type of entity
    type: string
    enum:
      - Person
      - Organization
  name:
    title: Name
    description: Name of the organization.
    type: string
    $linkedData:
      term: name
      '@id': https://schema.org/name
  legalName:
    title: Legal Name
    description: The official name of the organization, i.e., the registered company name.
    type: string
    $linkedData:
      term: legalName
      '@id': https://schema.org/legalName
  url:
    title: URL
    description: URL of the person or organization.
    type: string
    $linkedData:
      term: url
      '@id': https://schema.org/url
  taxId:
    title: Tax Identification Number
    description: >-
      The Tax / Fiscal ID of the organization or person, e.g., the TIN in the US
      or the CIF/NIF in Spain.
    type: string
    $linkedData:
      term: taxId
      '@id': https://schema.org/taxID

We could keep a small number of well-defined requirements, and then just say additionalProperties: true.

Really close - TIN will be optional in many cases or will be PII.

We will also want to have explicit (optional?) fields for address/place, phone, and email, and likely a DID (though in our case the DID would be the id value)

Then the additionalProperties will take care of the rest

Pinging others to make sure i am not missing something critical for defining a Person or a Business @rhofvendahl @brownoxford

[BenjaminMoe]

Sounds like

required:
- type
- entityType
- name
optional (defined):
- legal name
- tax id number
- address
- phone
- email
- id / did
- url(?)
additionalProperties: true

[BenjaminMoe]

Got into this a little bit, and I think I understand part of the problem. The Entity type is not structured in a JSON-LD friendly way.

$linkedData:
  term: Entity
  '@id': https://w3id.org/traceability#Entity
title: Entity
description: A person or organization.
oneOf:
  - $ref: ./Person.yml
  - $ref: ./Organization.yml
example: |-
  {
    "type": "Person",
    "firstName": "Kane",

In this case all Entity is doing is passing through to Person, or Organization, and I don't think that JSON-LD supports this behavior on a fundamental level. We would need something closer to the following.

type: object
  type:
    oneOf:
      - type: array
        items:
          type: string
          enum:
            - Entity
      - type: string
        const: Entity
  subject:
    description: The type of entity
    oneOf:
        - $ref: ./Person.yml
        - $ref: ./Organization.yml

In order to have an Entity node, and then have subject stem from that. Since all of the schemas filter down to this reference, even a change like this requires a lot of work. I'm going to work towards flattening the schema as I think that is the lowest risk solution at this point.

[mkhraisha]

There looks like there's a number of other oneOf not just in Entity, even if we flatten just entity we'll need to flatten the rest too no?

Also is this also an issue for Array vs String one of?

[BenjaminMoe]

I don't think it's an issue for array vs string for oneOf. So far it seems to be specific to the way Entity.yml was implemented as a pass-through without defining it's own node.

[BenjaminMoe]

Suggest moving this to its own issue:

Approach 4: Update JSON-LD processor to support oneOf schema functionality
Current Conclusion: No one has volunteered to implement this change

And then closing this issue.

[OR13]

Close this issue.

Open 2 new issues:

1. How to handle oneOf in the JSON Schema to JSON-LD context code.
2. How to define JSON-LD Classes so that no undefined terms are created related to "entityType".

[OR13]

link issues before closing.

[BenjaminMoe]

https://github.com/w3c-ccg/traceability-vocab/issues/527

[BenjaminMoe]

https://github.com/w3c-ccg/traceability-vocab/issues/528