Badge extensions proposal (draft 0.2)

ottonomy commented 10 years ago

Abstract

This proposal shows how additional metadata fields to add more information to a badge could be standardized in an open distributed fashion in keeping with the spirit of the Open Badges Infrastructure.

The Problem

The 1.0 Open Badges Specifcation allows issuers to include additional metadata fields in the badgeClass JSON, saying "additional properties are allowed if they don't clash with existing properties." Issuers are encouraged to namepace their additions to minimize the risk of two issuers defining differently implemented properties that have the same name.

This type of freedom makes it impossible to determine how to interpret the additional metadata, and there is no way to tell if two different badges use the same understanding of an additional property that they have included.

For example, two issuers include a property named "location" in their badge. One provides a URI of a website where someone can sign up for an online course that issues the badge. The other provides a US street address and geolocation of a school that at one time offered a course that issued the badge.

Discussion

Much of the usefulness of the additional properties, as currently allowed, is assumed to be internal to the issuer, because the implementation of those properties could be closed-source. But many badges might have use for providing more information in a common, machine-readable format.

The Proposal

Define an extensions array as a property of the badge class, issuer organization, and badge assertion JSON. This is an unordered list of extensionObjects, each of which follow a structure like the following example using JSON-schema (JSON-LD, a w3c-track similar structured data specification, is also under consideration):

{
    "schema": {
        "reference": "http://example.org/badgeExtension1v1.json",
        "hash": "sha1$cf23df2207d99a74fbe169e3eba035e633b65d94"
    },
    "content": {
        "extensionProperty": "some text"
    }
}

The proposal also requires an extension definition file to be accessible at the schema.reference URL (JSON-schema). In this case, it would define that the extension content could (or must) have a property called extensionProperty whose content would be text. The issuer generates a sha1 hash of the json retrieved from that location and includes it in the referrer object following the format alg$hash. This allows applications dealing in extended badges to cache them and determine if schema have changed, or if schema referenced at different locations are identical.

Consequences

Backwards Compatibility: The proposal is backwards compatible with the 1.0 assertion specification for the badgeClass. It is ambiguous if it is compatible with the assertion or the issuer organization, though the addition does not conflict with any existing property. If an application analyzing the extended badge does not know how to understand the extensions array, it should ignore and leave them intact.

Standardized extension formats allow multiple issuers to define additional metadata in common, so that badge consumers can better understand...

What the additional information means to the issuer in terms of what fields are available and/or required as well as the data types that could populate those fields.
Whether two different badges are using the same definition for an additional property as each other.
Whether a badge extension is properly formed according to its definition.
Request for Comment and Code
- We are interested in fleshing out the JSON-schema examples, including applications that verify example badge objects against their schema. Look through the pull requests in this repository for some ideas of how extensions could be used.

canweriotnow commented 10 years ago

Since we were talking today about xAPI integration as a desired extension (or evidence form), maybe looking at the xAPI extensions spec would be useful?

cmcavoy commented 10 years ago

Quick pingback to #8

cmcavoy commented 10 years ago

After offline discussion with @brianloveswords @kayaelle and @ottonomy, we're suggesting a change to the core proposal.

Issue with the current proposal

Putting extensions in a specific extensions namespace makes it difficult to 'graduate' an extension to a first class assertion specification member. For example, an experiment based on the original proposal in #8 :

{
  "name": "Awesome Robotics Badge",
  "description": "For doing awesome things with robots that people think is pretty great.",
  "image": "https://example.org/robotics-badge.png",
  "criteria": "https://example.org/robotics-badge.html",
  "tags": ["robots","awesome"],
  "issuer": "https://example.org/organization.json",
  "alignment": [
    {
      "name": "CCSS.ELA-Literacy.RST.11-12.3",
      "url": "http://www.corestandards.org/ELA-Literacy/RST/11-12/3",
      "description": "Follow precisely a complex multistep procedure when carrying out experiments, taking measurements, or performing technical tasks; analyze the specific results based on explanations in the text."
    },
    {
      "name": "CCSS.ELA-Literacy.RST.11-12.9",
      "url": "http://www.corestandards.org/ELA-Literacy/RST/11-12/9",
      "description": " Synthesize information from a range of sources (e.g., texts, experiments, simulations) into a coherent understanding of a process, phenomenon, or concept, resolving conflicting information when possible."
    }
  ],
'extensions': [
{
    "schema": {
        "reference": "http://example.org/badgeExtension1v1.json",
        "hash": "sha1$cf23df2207d99a74fbe169e3eba035e633b65d94"
    },
    "content": {
        "age": "K-12",
        "location": {
           "url": "http://online-course",
           "streetAddress": "123 test rd",
           "city": "providence",
           "stateProvince": "RI",
           "zipPostal": "02903"
      }
    }
}
]
}

If the extensions example above is put into the wild, a client library would need to understand the badgeSpecification referenced in the extension url. When the extension is accepted into the assertion standard, clients will need to be rewritten to understand the new location.

Revised Proposal (0.3)

Extensions are top level members of the assertion. A new field, badgeSpecifications is added to the assertion that provides an array of specification urls.

An example,

{
  "name": "Awesome Robotics Badge",
  "description": "For doing awesome things with robots that people think is pretty great.",
  "image": "https://example.org/robotics-badge.png",
  "criteria": "https://example.org/robotics-badge.html",
  "tags": ["robots","awesome"],
  "issuer": "https://example.org/organization.json",
  "alignment": [
    {
      "name": "CCSS.ELA-Literacy.RST.11-12.3",
      "url": "http://www.corestandards.org/ELA-Literacy/RST/11-12/3",
      "description": "Follow precisely a complex multistep procedure when carrying out experiments, taking measurements, or performing technical tasks; analyze the specific results based on explanations in the text."
    },
    {
      "name": "CCSS.ELA-Literacy.RST.11-12.9",
      "url": "http://www.corestandards.org/ELA-Literacy/RST/11-12/9",
      "description": " Synthesize information from a range of sources (e.g., texts, experiments, simulations) into a coherent understanding of a process, phenomenon, or concept, resolving conflicting information when possible."
    }
  ],
"age": "K-12",
"location": {
      "url": "http://online-course",
      "streetAddress": "123 test rd",
      "city": "providence",
      "stateProvince": "RI",
      "zipPostal": "02903"
 },
'badgeSpecifications' : [
    'http://openbadges.org/assertionSpecification1.0.json', 
    'http://example.org/badgeExtension1v1.json'
]
}

The example above adds an age and location member to the assertion. The schema definition for the additions is at http://example.org/badgeExtension1v1.json and is referenced in badgeSpecifications. A bonus feature of linking to extensions in this way is that the core assertion specification becomes a linked badgeSpecification.

When the extension described in http://example.org/badgeExtension1v1.json is accepted into the core assertion specification, the change is reflected in the revision number of the core assertion specification. The badgeSpecifications in the above example would change to

'badgeSpecifications' : [
    'http://openbadges.org/assertionSpecification1.1.json'
]

kayaelle commented 10 years ago

This is good. I like how the reference to the openbadges spec is in here along with the extension spec.

One question: should there be two extension references? One for age and one for location?

cmcavoy commented 10 years ago

@kayaelle - I think an extension spec can define multiple new members. Maybe age and location don't belong together...but...if they did, they could be in a single specification.

kayaelle commented 10 years ago

I think we may want to be careful about combining specs. I think it can diminish the effectiveness of the idea of extensibility if they share specs. If specs get combined, there should probably be a reason for combining vs not combining.

canweriotnow commented 10 years ago

@kayaelle I think there should... It makes sense to me that top level elements should be individually validatable if they're not in the core assertion spec.

ottonomy commented 10 years ago

@kayaelle I've been starting some work on a node module that will do verification of badge objects with top-level property extensions. It leads me to do a lot of thinking about what is the closest "go-with-the-flow" structure for schema verification, and I agree that one schema specification per additional property is best.

I think from this revised draft, we'll probably need to modify the contents of badgeSpecifications to not be bare URLs, because I'd like to know as I'm beginning the verification process which property to test against which specification.

One possibility would be to use a list of properties instead (object format):

'badgeSpecifications' : {
    'obi-badgeclass':'https://openbadges.org/1.1/badgeClassSpecification.json', 
    'location':'https://example.org/badgeLocationExtension1v1.json',
    'age':'https://example.org/badgeAgeExtension1v1.json'
}

Or an array of objects:

'badgeSpecifications' : [
    { 'name':'obi-badgeclass',
    'reference':'https://openbadges.org/1.1/badgeClassSpecification.json' }, 

    { 'name':'location',
    'reference':'https://example.org/badgeLocationExtension1v1.json' },

    { 'name':'age',
    'reference':'https://example.org/badgeAgeExtension1v1.json' }
]

...and one element of the 0.2 proposal I would like specific comment on would be the 'schema-hash' component, which would be compatible with this second approach. The advantages from the perspective of the validation software package being:

Validator can tell if the badge object at hand is truly using the same version of the badgeSpecification as had been previously cached
Validator can tell if schemas at different reference URLs are identical
Validator can tell if an extended badge using a badge extension subsequently moved into the standard is following the same spec as was standard-approved.

Disadvantages being:

Significantly more complicated for issuers and issuing software.

canweriotnow commented 10 years ago

@ottonomy :+1:

ottonomy commented 10 years ago

@cmcavoy: I've been thinking a bit on your comment from today's BA Standard Working Group call on whether or not to reference which property name is attached to a particular extension schema, and it could indeed be done either way. Some thoughts:

Your 0.3 proposal way would ask a verifier to pass the whole badgeClass through the badgeClass schema (this schema would allow additionalPropeties),then pass the same thing through each successive badgeSpecification (which would each only define the additional properties that it was concerned with and again allow additionalProperties besides the ones it "knows" about)
Whereas my way would pass the whole badgeClass through the badgeClass schema, then filter out individual properties to pass through the rest of the specified schema. In some ways this is more complicated, because the badgeSpecifications list would contain two different types of schema, one that applies to the whole badgeClass, and one that applies only to an individual property. It would also prevent specifications from being multi-property in nature. (Aside: what is the killer app for allowing multi-property extensions rather than making people package their extensions one property at a time?)
Both options require a verifying party to keep track of which properties have and haven't been successfully passed through a schema for structure verification. The same sort of logic to allow a verifier to handle extensions moved into the core standard since the subject badge's issuance would apply to both.
I was trying to imagine a case where there are two competing "location" specs trying to get into the standard. Say, last year's schema http://example.org/location.json does it by geo coordinate and a newer option http://other.org/location.json allows both geo coord and US street address. An issuer wants their badges properly understood by applications only trained to understand last year's extension, but also wants to include the new features of the new schema. Unfortunately, both schema suggest using the same property "location" to contain the extra metadata. The 0.3 approach wouldn't be able to handle this case.
Tradeoffs on each side.

ottonomy commented 10 years ago

Ok, @cmcavoy your draft 0.3 is fine by me. :thumbsup:

As you define it, additional properties may be added to a badge object:

"photographOfAPrintedCopyOfTheBadgeSignedByTheUniversityPresident": {
    "imageUrl":"http://issuer.edu/badges/signature-images?uid32jf9l3"
}

and a schema array must be referenced:

"badgeSpecifications": [
    "https://openbadges.org/standard/1.5.json#/badgeClass",
    "https://issuer.edu/badges/extensions/photographOfPrezSigv1.0.json"
]

The schema (in JSON-schema) for each badgeSpecification tells you which properties it covers, starting with the OBI schema, which defines all the the core properties, allowing additionalProperties. Every other badgeSpecification must allow additionalProperties as well (allowing the core properties without redefining them). Each schema is designed to tell consumers how to interpret the additional properties the issuer wants to use How could you use

{
"$schema":"http://json-schema.org/draft-04/schema#",

"definitions": {
    "prezSig": {
        "type":"object",
        "properties": {
            "imageUrl": { "type": "text", "format": "uri"}
        },
        "required": ["imageUrl"],
        "additionalProperties":"false"
    }
},

"type:":"object",

"properties":{
    "photographOfAPrintedCopyOfTheBadgeSignedByTheUniversityPresident": {
    "$ref": "#/definitions/prezSig"
    }
},
"required":["photographOfAPrintedCopyOfTheBadgeSignedByTheUniversityPresident"],
"additionalProperties":"true"
}

This schema, hosted at the second badgeSpecification address listed in the issuer's badgeClass, only concerns itself with properties added to the badge, allowing any other properties of the badgeclass to be governed by other rules, in this case, the rules of the Open Badges Standard 1.5 badgeClass schema.

I'm attached to the idea that we could get several advantages from issuers providing a hash of the schema for consumers' and the ecosystem's benefit, but I haven't managed to convince anybody. Sounds like the right situation for an extension, right?

We could add a second property that could contain metadata on each of these specifications, including my hash function:

"badgeSpecificationsSecure": {
    "https://openbadges.org/standard/1.5.json#/badgeClass":{
        "target": "#",
        "hash": "sha1$ecf6446d4b181abe9e2b173ab5e1f9a47558a7bf"
    },
    "https://trustworthybadger.org/standard/photographOfPrezSigv1.0.json": {
        "target": "#/photographOfAPrintedCopyOfTheBadgeSignedByTheUniversityPresident",
        "hash":"sha1$0b7303a037d2e108bae11b83b92e5d1585b8a5c8"
    },
    "https://ottonom.net/extensions/badgeSpecificationsMeta0.5.json":{
        "target": "#/badgeSpecificationsSecure",
        "hash": "sha1$0a493fe48e34bd0a7610da0cfdfce9ad32a2ee11"
    }
}

Each extension might define rules for multiple properties, and @cmcavoy's proposed technique doesn't help us recover from conflicts in how different extensions may define a particular property. Yeah, that's pretty esoteric and unlikely. Perhaps.. Maybe in the future, issuer.edu decides that they want to use an updated version of the extension that allows a second property, "presidentFullName":

"photographOfAPrintedCopyOfTheBadgeSignedByTheUniversityPresident": {
    "imageUrl":"http://issuer.edu/badges/signature-images?uid32jf9l3",
    "presidentFullName": "Mary Q. Edie"
}

This uses the same property name as before but would fail verification against the original schema which didn't allow additional properties inside "photographOfAPrintedCopyOfTheBadgeSignedByTheUniversityPresident". So they add an additional property "prezPhotoUpdate" to the badge that is a duplicate other than adding the new property.

Perhaps "badgeSpecificationSecure" could allow issuers to declare exceptions to rules found in the schema:

"badgeSpecificationsSecure": {
    "https://openbadges.org/standard/1.5.json#badgeClass":{
        "target": "#",
        "hash": "sha1$ecf6446d4b181abe9e2b173ab5e1f9a47558a7bf"
    },
    "https://trustworthybadger.org/standard/photographOfPrezSigv1.0.json": {
        "target": "#/photographOfAPrintedCopyOfTheBadgeSignedByTheUniversityPresident",
        "hash":"sha1$0b7303a037d2e108bae11b83b92e5d1585b8a5c8"
    },
    "https://trustworthybadger.org/standard/photographOfPrezSigv2.0.json": {
        "target"[: "#/photographOfAPrintedCopyOfTheBadgeSignedByTheUniversityPresident", "prezSig2"],
        "renamedProperties": {
            "#/prezSig2": "#/photographOfAPrintedCopyOfTheBadgeSignedByTheUniversityPresident":
        },
        "hash":"sha1$df403b732380ce32dac0e396ad076f92c15498d7"
    }
    "https://openbadges.org/extensions/badgeSpecificationsMeta0.5.json":{
        "target": "#/badgeSpecificationsSecure",
        "hash": "sha1$0a493fe48e34bd0a7610da0cfdfce9ad32a2ee11"
    }
}

And then as an extension author, my responsibility would be to give the issuers good enough benefits for using the extension that they'd give me the schema-hashes I would like to have as a developer of badge consumer applications. And I've got to build the consumer applications that would give them and other consumers to exploit the value of this possible extension. It's possible nobody will care about these features, and the OB ecosystem would still be better off for having added the extension features (and the additional flexibility of @cmcavoy's approach that my 0.2 proposal did not have.)

Ok, @cmcavoy, I am on board with your style of doing it:

"badgeSpecifications": [
    "https://openbadges.org/standard/1.5.json#badgeClass",
    "https://issuer.edu/badges/extensions/photographOfPrezSigv1.0.json"
]

cmcavoy commented 10 years ago

@ottonomy don't give up just yet. There's a lot of material in this thread, need to think it through a bit more. I'd really like to understand more about json-schema. A lot of this is riding on json-schema working well for us, but none of us have significant experience with it (that I know of).

kayaelle commented 10 years ago

Andrew seems pretty familiar with json-schema. Could we bring him in to help thing through some things?

kayaelle commented 10 years ago

Helpful Lint tool: http://jsonschemalint.com/

kayaelle commented 10 years ago

npm validator module https://www.npmjs.org/package/jsonschema

ottonomy commented 10 years ago

I pinged @andrewhayward on IRC -- maybe we'll be lucky enough to get him to drop in.

I've been playing around with JaySchema, which is schema draft 4 ready, but that was the only factor I looked at when I chose one to fiddle with.

@kayaelle, thanks for the lint tool!

andrewhayward commented 10 years ago

That I'm aware of, there's no way to handle dynamic schema validation based on properties unknown in advance. It's not a problem in and of itself, but would require us to write our own custom solution for validation (which it seems @ottonomy has already started work on anyway), rather than just using any old JSON schema validator.

The only solution I can see that would stay within the bounds of pure JSON schema validation would be to have entirely custom schemas that extend the core definition as provided by the OBA.

{
    "$schema": "http://json-schema.org/draft-04/schema#",
    "title": "Badge Class Extension",
    "type": "object",
    "allOf": [
        {"$ref": "http://openbadges.org/schema/v1.1/badge-class.json#"},
        {
            "properties": {
                ...
            }
        }
    ]
}

Anyway, a suggestion, based on reading through the above (though one I have my reservations about):

A badge-class-specification schema is defined, that any extension (including the core badge-class schema that the OBA writes) must validate against. This would essentially be a subset of the core schema, but would allow certain properties to be controlled (such as preventing things like "additionalProperties": false).
Any specifications found in the given list must validate against this new schema - the definition would be considered invalid if not.

The downside to this idea is that any existing JSON schema already written (such as geo) would have to wrapped in a second 'badge specification' compliant schema, causing more work for both spec writers, and potentially the validator. You would also have to ensure that (some version of) the core specification is actually listed - what happens if it isn't? Is it still a valid badge?

Based on this final point, I would instead recommend not including the badge spec in the list, and rather introduce two new properties: version and extensions (or similar). Any badge not having a version attribute would be considered to be v1.0 compliant, and treated as such. extensions would be like the aforementioned badgeSpecifications list, but only include references that add properties to the core specification. A badge-class-extension schema could still be created that these would have to comply with.

I don't know if it makes more sense to just have a list of documents that extend the spec, or to explicitly itemise every new property, and a location at which it is defined ({"location": "http://example.org/spec.json#/geo"}, for example). I'm also not sure if you'd want to disallow any additional properties not defined by one extension or another.

Side note - I'd err on the side of putting something like a $ on the front of whatever property you decide to use, like $badgeExtensions, to prevent collisions.

timothyfcook commented 7 years ago

Moving to archive.

1EdTech / openbadges-discussion