Recognition levels for a QACredentialOrganization

[jeff-grann]

Example from Strong Workforce Stars uses three levels with Bronze, Silver, & Gold levels.

[stuartasutton]

The Strong Workforce Stars are associate with "programs" in the California Community College system by its Chancellor's Office that meet certain statistical criteria. All of our current forms of QA "recognition" are binary--awarded/not awarded. This introduces an open, infinitely variable form of "recognition" by QA orgs (formal and informal) requiring, at a minimum, information about name of recognition, type of recognition, criteria of award. For the credential, learning opportunity (program) or assessment(?) receiving such recognition would also not be binary statement.

Since this suggests another profile, more evidence needed--how far from being an edge case is this? We'd need a number of examples (10?) to know that any profile devised is covering the bases.

[siuc-nate]

This introduces an open, infinitely variable form of "recognition" by QA orgs (formal and informal) requiring, at a minimum, information about name of recognition, type of recognition, criteria of award.

This is exactly why we decided that QA would be a (type of) Credential, so we could use all of the fields that Credential captures. We had originally envisioned a QA Profile, but as we delved further into the use cases it would need to cover, it began to overlap almost entirely with Credential. Making it a type of Credential simplified a lot of things.

QA Organization -offers-> Gold Star Credential
QA Organization -offers-> Silver Star Credential
QA Organization -offers-> Bronze Star Credential

The trick is not to get distracted by the implied linear scale of QA being offered - those could just as easily be called "Super amazing quality award A", "B", and "C" - the thing to keep in mind is that each is a standalone credential, independent of the others, with its own set of requirements and information, and each could be awarded to some other entity because the QA Organization offers it.

[stuartasutton]

@siuc-nate, first, I didn't suggest a QA profile. I was asking about a recognitions profile. Second, these aren't three credentials--Gold, Silver, Bronze as your solution suggests. They are a fleeting characteristic of a QA judgement about a program and not a credential type. The basic thrust of my comment was that QA decisions/recognitions so far in the CTDL are binary, these are nuanced.

[siuc-nate]

I didn't suggest a QA profile

This was my interpretation of

Since this suggests another profile

Though at this stage the name of it probably doesn't matter. Anyway, in this case, each of the stars is tied to some set (or rather, some number) of requirements (e.g. "meet x, y, and z to get a gold, x and y to get a silver, and z for a bronze").

Unless the QA being awarded is completely arbitrary (like user ratings) then I don't see how the notion of a QA credential doesn't fit here. This also solves it for broader scopes/use cases beyond this particular one.

[stuartasutton]

Nate, your example above clearly does not work. The provider offers one credential that a 3rd party (QA) algorithmically rates and assigns a star according to it's criteria...and that program could lose it the next year and gain it back the following. So, do it from the rated program view...how does a program say that it received 2 stars in the CCCO ratings and that those were for criteria 1 & 2 but not 3).

[siuc-nate]

We need to avoid conflating the generic description of the credentials with the description of the earned/issued credentials. However, if we're factoring in the issued/awarded data as well, then you'd use RecognizeAction to convey the instance-related data, where:

{
  "@type": "ceterms:RecognizeAction",
  "ceterms:actingAgent": "URI to QA organization",
  "ceterms:object": "URI to recipient of QA",
  "ceterms:instrument": "URI to star credential",
  "ceterms:evidenceOfAction": "URI to something that indicates which criteria were met"
}

In other words, you handle the problem of something infinitely variable that's beyond our ability to predict and beyond our scope by handing it off to the evidenceOfAction URI, thus enabling the publisher to convey detailed information in their own way (via webpage, JSON, whatever suits their needs).

If you do that, then the number of credentials offered becomes almost irrelevant, as long as there's at least one. One credential could encompass all three criteria and mention that the more of them that you meet, the better you will be rated (though this does mean that no organization could, via URI, claim to have been awarded a "gold star" vs a "bronze star").

Or they could do what I recommended above and offer one credential for each "star level", so that the URI to that particular star level could be pointed to by the RecognizeAction (which would make it easier to consume/search/compare/etc. the data).

Whether or not Action subclasses like that belong in the Registry (or in some other repository) isn't something that we've really discussed, but that is probably a separate issue.

[siuc-nate]

The evidence indicating which criteria were met could (and probably should) also be furnished by the data on the other ends of the URIs in Verification Service Profile, namely Verification Directory and Verification Service. That would be independent of the data above, but it could use the same class (RecognizeAction) with the same data internally.

And actually, given that there is real and potentially immediate weight/importance tied to this kind of QA, it probably makes sense to keep it maintained within the QA organization's own database, rather than risk them getting out of sync in some registry.

[siuc-nate]

I dug up this old diagram from back when we were designing the system to work with the notion of a QA credential and the various Action profiles: The intent here is to convey:

• 1 (blue bar): An organization indicates that it offers accreditation, generally, as a "service" (that is to say, "our QA organization accredits other things" without specifying any other "thing". This happens via the ceterms:serviceType property and ceterms:AgentServiceType vocabulary.
• 2 (green bar): An organization indicates that it accredits some other organization, without specifying anything about that accreditation (i.e., what standards were met, the date of issuance, the duration, etc.). This happens via the ceterms:accredits property (and the reverse via the ceterms:accreditedBy property).
• 3 (orange bars): An organization indicates, via a ceterms:AccreditAction, that it has issued a specific form of quality assurance (the ceterms:QualityAssuranceCredential) on a specific date to a specific organization.

All of the information that has to do with the instance of the issued QA is embodied in the Accredit Action. The Accredit Action points, among other things, to the generic, non-issued, abstract description of the QA (including the requirements, a description of what it means, etc.) that is embodied in a standalone ceterms:QualityAssuranceCredential.

This is akin to an organization saying "we gave abstract description of a degree to person on date" where to person on date is a completely separate piece of information from the abstract description of a degree itself and embodied in a separate container (equivalent to our Action classes).

[stuartasutton]

@siuc-nate and @jeff-grann , I'm signaling @jkitchensSIUC here because this is the first foray into use of one of the subclasses of ceasn:CredentialAction. Nate, I think that your figure above is indeed an "old diagram"...and a bit off. You were correct back up a few posts on this issue that the proper class of action is ceasn:RecognizeAction since it is the Chancellor's Office of the California Community College System recognizing a program of study --for illustration here, the SRJC Dental Assisting Program recognized with 3 stars in the "2018 Strong Workforce Stars" program of CCC system-wide recognitions.

The correct modeling of the recognition action is as follows:

Note that in the ceasn:RecognizeAction class there is no property for pointing to any sort of "award" entity. This is true of all of the ceasn:CredentialAction subclasses except for the ceasn:AccreditAction and ceasn:RenewAction classes which have a ceasn:resultingAward property. The ceasn:resultingAward property has a range of ceasn:CredentialAssertion--i.e., an awarded credential. To date, we have not defined any properties for ceasn:CredentialAssertion since we have, so far, totally shied away from representing awarded anything as entities in the CER.

So, if the need here is pressing in terms of work with the CCC and taking what I note above to be correct, we need to get this on a Wednesday agenda for CTDL review for discussion when Jeanne is present. Are we otherwise prepared in terms of ingest, validation, editor and documentation to handle any of these enabled actions?

[siuc-nate]

The property used to point to the "award" entity is ceterms:instrument, defined (rather generically - borrowed from schema.org if I recall) as: "Object that helped the agent perform the action. e.g. John wrote a book with a pen." With a comment that is far more relevant: "A credential or other instrument whose criteria was applied in executing the action."

The ceterms:evidenceOfAction would also point to something other than the QA organization: Definition: "Entity that proves that the action occured or that the action continues to be valid." Comment: "The evidence verifies the information in the action and is particular to it. It is not a directory of such evidentiary entities or a description of how such verifications might generically be characterized." Its range is xsd:anyURI, so it's very flexible in terms of what could be considered "evidence" that some entity had received an instance of the QA credential.

Or, to represent it visually:

Which, if you include some more CTDL properties not directly related to the Action class, becomes:

And if you flesh it out a bit further (green arrows) and include things we don't have control over (blue arrows):

Edit: Fixed a typo in the diagrams

[stuartasutton]

Nate, I take massive issue with the notion of the existence of "Silver-", "Bronze-" and "Gold" awards instantiated as QA Credentials and that the Chancellor's Office offers them when all they do is recognize a program and where every means is available to handle that without creating a thing that cannot be recognized as anything other than a ceasn:CredentialAssertion.

Also, instrument and evidenceOfAction can point to pages on the ActingAgent website where they can be found.

[siuc-nate]

If they create a QA Credential for each of the ranks, they can reuse them by pointing to them from the actions (where there one action for each thing being recognized). They would have to duplicate the data if they tried to pack it all into CredentialAssertions.

As for ceterms:instrument, the range doesn't include xsd:anyURI - it can only point to subclasses of Credential.

This particular use case is maybe not the best to examine in terms of our design, as a lot of the solutions are probably overkill.

Maybe a good solution for this specific case would be to rely solely on RecognizeActions and use the ceterms:evidenceOfAction property to point a page that describes which particular combination of criteria were met by that organization to enable it to reach Gold/Silver/Bronze status?

If I may go off on a bit of a tangent related to that: It is a bit of a curve ball in that the statuses are awarded to things that meet any 1/2/3 of the 3 criteria, rather than tying them to specific criteria explicitly. However, it would be relatively trivial to use URL parameters to tell a single web page which combination of criteria to display. Add another parameter that causes the page to show something relevant to the particular recipient of the award and the page could also then technically be evidence, making it semantically correct to use it as the value for evidenceOfAction property.

[stuartasutton]

Yes, your suggested solution is overkill and: (1) a massively slippery slope; and (2) a degradation of the notion of QACredential that would lower user confidence in CER serving its primary purpose. Going down that slippery slope would fill the registry with such recognitions. Consider also that there's little difference substantive between the CCC's Workforce Stars "recognition" of a credential, learning opportunity, assessment as QACredentials and an employer's "recognition" of a credential, learning opportunity, assessment as QACredentials.

One other point. If we instantiate such recognitions as the CCC's Workforce Stars as an array of QACredentials, why do we bother instantiating a "RecognitionAction" at all?

[stuartasutton]

The ceterms:instrument range should be changed to xsd:anyURI to more fully comply with the comment which goes beyond credentials:

A credential or other instrument whose criteria was applied in executing the action.

That would include competency frameworks, assessment rubrics etc.

Of course xsd:anyURI would include the current array of ranges, so it's backward compatible.