Are controlled vocabularies new standards or part of the standard containing their properties?

[baskaufs]

@tdwg/exec @tucotuco (Darwin Core) @pzermoglio (BDQ TG4)

I was reviewing the timeline we've set as a goal for the development of the controlled vocabularies for the Audubon Core terms ac:subjectPart and ac:subjectOrientation and it occurred to me that I don't actually know how the end game for this process plays out. There are two ways of thinking about this task:

1. We are creating controlled vocabularies that are new standards falling into the category "Data Standard (DS)", described on the Standards Status and Categories page as "Specifies valid values in controlled vocabularies".
2. We are creating controlled vocabularies that are coordinated additions to an existing standard (Audubon Core) since the vocabulary terms are intended to be used as values for property terms in Audubon Core (ac:subjectPart and ac:subjectOrientation).

Since TDWG has not yet adopted any controlled vocabularies, we do not have any precedent for how to treat this situation. There are significant implications based on the choice that is made.

If we are creating a new standard, then the governance of the process falls under the "Ratification of standards" process described in the TDWG Process document. In that process, the Executive Committee generally controls the process by deciding when the work has advanced to the stage where a Review Manager should be appointed. The Review Manager then coordinates the public comment process and revisions, with a final decision by the Executive.

If we are creating an addition to an existing standard, then the governance of the process falls under Section 4 of the Vocabulary Maintenance Specification (VMS) since the controlled vocabulary is a coordinated set of additions to the vocabulary (as opposed to an addition of a single term). In that process, the Maintenance Group (MG) responsible for the existing vocabulary standard (the vocabulary containing the property term for which the new controlled vocabulary serves as values) generally controls the process. By examining the Feature Report and Implementation Experience report, the MG determines whether the proposed enhancement (the controlled vocabulary) meets the requirements of Section 3.1 -- in particular the efficacy requirement as evidenced in the Implementation Experience report. If those requirements are met, the MG advances the proposed enhancement to public comment, followed by revisions and ending with a final decision by the Executive.

So there are two key differences. One is that VMS has additional requirements (the Feature Report and Implementation Experience Report) that are not part of the generic standards Process. (I would, however, argue that those reports SHOULD be part of any vocabulary development, since they are modeled after the best practices of our aspirational peer organizations like W3C and IETF. The Views TG that I'm convening is creating them regardless of whether we are required to or not.) The other is that in the generic Process an external review manager manages the process and in the VMS process the MG manages the process.

Aside from the philosophical considerations of whether the controlled vocabulary is a new standard or an addition to an existing one, there are a couple practical considerations. In the case of the two Audubon Core controlled vocabularies (values for ac:subjectPart and ac:subjectOrientation), the convener of the Task Group developing the controlled vocabularies and the convener of the Maintenance Group who would be in charge of the review process under the VMS is the same person (me). That seems like it would be a significant conflict of interest. The other practical issue is bandwidth. Given that there are only two related controlled vocabularies under development at the moment for Audubon Core, the AC MG would probably have the bandwidth to manage the process. But there are multiple controlled vocabularies being developed for Darwin Core (three just related to dwc:establishmentMeans, plus who knows how many more other ones being worked on by BDQ TG4). If the DwC Maintenance Group has to manage the evaluation of Implementation Reports, public comments, and suggesting revisions as suggested in the VMS for several controlled vocabularies at the same time, that's a big ask.

One possibility is to treat the controlled vocabularies as new standards and have an external review manager manage the comment period and revisions as described in the generic Process document. The new vocabulary standards could then be assigned to the existing Maintenance Group (DwC or AC) to maintain them after they are completed. Section 2.1 of the VMS specifically allows for this possibility:

In the case where a new vocabulary is related to an existing vocabulary having an existing maintaining Interest Group, maintenance of the new vocabulary may be assigned to the existing Interest Group. Several minor vocabularies may be maintained as a group by a single Maintenance Interest Group. The decision of maintenance responsibility for each vocabulary standard rests with the Executive Committee.

At least in the case of the Audubon Core controlled vocabularies we are at least a year away from needing to know how to handle this, but for some of the DwC controlled vocabularies we may be a lot closer to review than that. So proactively having a policy decision on how to handle this when it come up would be a good thing.

[qgroom]

This would be an excellent subject for the upcoming TDWG Exec meeting in Leiden. I hope @jmacklin can find some time for it in the agenda. Personally I'm relaxed on the issue of a conflict of interest (you might say I have a conflict of interest in saying that too). Nevertheless, finding review managers will be difficult and no one has more motivation to stick by the process than the proposer themselves. I think transparency, oversight and sign-off are enough to balance the conflict.

Regarding, the issue of new standard verses maintenance, I tend towards maintenance. Few people understand the nuances of these standards better than the maintenance group.

[tucotuco]

This is certainly a critical thing to resolve in light of the plans for the BDQ Task Group 4 on Vocabularies of Values. Another interesting twist is the effort to nascent effort to reconcile Darwin Core and ABCD. Those two standards have a LOT of concepts in common, though they do not share terms. They will certainly want vocabularies of values in common. While they remain separate standards, however, they will have their own maintenance groups, and therefore the strategy to have the vocabulary of values development happen in the maintenance group would be complicated.

[jmacklin]

This is definitely an important conversation to have given the active work being done on vocabularies. I will indeed add this to the agenda of the Exec at the upcoming meeting in Leiden. I am also relaxed about the conflict issue. I do think we need to keep this process as light as possible. We really struggle to have enough volunteers to keep these processes moving and finding outsiders to help as reviewers, etc. makes things all the much harder... Have we considered having two processes for vocabularies? One that is very formal where the vocabulary is "mandatory" for a defined use versus ones that are more best practices that can still be "reviewed and approved" by TDWG but in a lighter sense. This would help with vocabularies that change and grow dynamically as we don't want to have to review versions on a regular basis. I also lean toward the Maintenance path but if we do this we need to provide training to a larger number of people so the burden does not end up lying on the few! I also see some of these issues having direct ties to our strategic planning exercise as we consider what TDWG aspires to be and how we can sustain our vision ;-)

[baskaufs]

There was no definitive answer to this issue. It appears that the process may depend on the nature of the controlled vocabulary (i.e. whether it is specific to properties in one parent vocabulary or whether it might be used by several). In the case of the vocabularies for ac:variant and dcterms:format, Audubon Core would be the primary user, so we are moving forward under the assumption that the AC Maintenance Group will manage their ratification and maintenance.