Vocabularies umbrella issue #17

Closed dpancic closed 1 year ago

In GitLab by @laureD19 on May 19, 2021, 14:12

this issue lists the concept-based properties found in the source of truth. Separate issues (and a dedicated file in the repo when needed) will be created for each of the vocabularies used:

[x] activity using TADIRaH (#4) -> https://gitlab.gwdg.de/sshoc/vocabularies/-/tree/master/tadirah
[x] geographical-availability using EOSC vocabulary (#9 )
[x] language using vocabulary: iso-639-3 (#5 )
[x] license using vocabulary software-license (#7 ) -> https://gitlab.gwdg.de/sshoc/vocabularies/-/tree/master/licenses
[x] life-cycle-status using EOSC vocabulary (#10 ) ->
[x] mode-of-use usine vocabulary: invocation-types (SSHOC WP3) -> https://gitlab.gwdg.de/sshoc/vocabularies/-/tree/master/invocation-types
[x] object-format vocabulary using iana-media-type (#11 ) ->
[x] publication-type using https://marketplace-api.sshopencloud.eu/api/vocabularies/publication-type (#20)
[x] technology-readiness-level using a mix between EOSC and CESSDA concepts (#12 ) ->
[x] discipline using subset of ÖFOS classification (#8 ) ->
[x] service-type vocabulary to be determined (#13) ->
[x] intended-audience using Training Discovery Toolkit and EOSC (#14) ->
[x] standard using SSK vocabulary (#15)
[x] EOSC-resource-category using EOSC categories (#16) ->
[x] keyword turning the values we have into concepts (#18 )

notify @KlausIllmayer @vronk @vronk @egray523 @vronk @vronk

In GitLab by @laureD19 on May 19, 2021, 14:12

added to epic &8

In GitLab by @laureD19 on May 19, 2021, 14:15

marked the checklist item activity using TADIRaH (#4 ) -> https://gitlab.gwdg.de/sshoc/vocabularies/-/tree/master/tadirah as completed

In GitLab by @laureD19 on May 19, 2021, 14:15

marked the checklist item language using vocabulary: iso-639-3 (#5 ) as completed

In GitLab by @laureD19 on May 19, 2021, 14:15

marked the checklist item license using vocabulary software-license (#7 ) -> https://gitlab.gwdg.de/sshoc/vocabularies/-/tree/master/licenses as completed

In GitLab by @KlausIllmayer on May 20, 2021, 17:45

We should also think about the workflow for updating/extending vocabularies.

It seems to be that there are three types of vocabularies:

Internal, open vocablaries like keywords: adding concepts on the fly (needs some way of managing the concepts e.g. merge same concepts) - the question would be, if for this vocabularies we need a workflow for adding new concepts? It could be argued, that they develop in a kind of folksonomy that is tidied up ever now and then by moderation (merging concepts, deleting ones that are not helpful/seldom used, etc.)
External, closed vocabularies like iso-639-3: they are not touched by MP. If there is an update of them, we can ingest the new vocabulary (after checking if the new version does not harm already tagged items)
Internal/external, controlled vocabularies like invocation-types: we steadily update them, they are maintained in MP and there is a workflow that defines how to add/remove concepts. External vocabularies are forked from the original one, we take the current version and develop it further in MP (that implies that an update on the external source may not be applyable anymore on the extended version in MP)

The workflow for the controlled vocagularies could look like this:

A concept is identified that is not present in a vocabulary
The concept is proposed to be added to a vocabulary (in the MP frontend or on a dedicated vocabulary management system?)
Responsible person for vocabularies accepts or refuses the proposal (what happens in case of refusal? will there be a kind of notification or some logging, so that others are aware about already refused concepts?)
If accepted, the new concept is added in the vocabulary management tool (PoolParty Thesaurus Server)
The changed vocabulary is exported as SKOS
The SKOS export is put into the MP backend by API call /api/vocabularies/{name-of-vocabulary}

One remark regarding the open vocabularies: I don't see currently an API call to add new concepts on the fly. So we would need to create this feature in the backend. For the frontend we will need some mockups how to handle new concepts (it could be useful for e.g. keywords, that users are asked if they really like to add a new concept including proposals for alternative concepts).

In GitLab by @laureD19 on May 27, 2021, 14:35

marked the checklist item mode-of-use usine vocabulary: invocation-types (SSHOC WP3) -> https://gitlab.gwdg.de/sshoc/vocabularies/-/tree/master/invocation-types as completed

In GitLab by @laureD19 on May 27, 2021, 14:47

marked the checklist item publication-type using https://sshoc-marketplace-api.acdh-dev.oeaw.ac.at/api/vocabularies/publication-type as completed

In GitLab by @laureD19 on May 27, 2021, 16:35

Update of the ingestion documentation regarding vocabularies (what is needed to import a ttl file in PoolParty for example) to be done here

In GitLab by @egray523 on May 27, 2021, 19:03

concerning the publication-type using https://sshoc-marketplace-api.acdh-dev.oeaw.ac.at/api/vocabularies/publication-type did we finally decide to not add other categories such as theses/dissertations or data papers/data journals?

In GitLab by @KlausIllmayer on Aug 2, 2021, 11:16

marked the checklist item geographical-availability using EOSC vocabulary (#9 ) as completed

In GitLab by @KlausIllmayer on Aug 2, 2021, 11:22

marked the checklist item life-cycle-status using EOSC vocabulary (#10 ) -> as completed

In GitLab by @KlausIllmayer on Aug 2, 2021, 11:24

marked the checklist item object-format vocabulary using iana-media-type (#11 ) -> as completed

In GitLab by @KlausIllmayer on Aug 2, 2021, 11:27

marked the checklist item EOSC-resource-category using EOSC categories (#16) -> as completed

In GitLab by @KlausIllmayer on Aug 2, 2021, 12:35

marked the checklist item standard using SSK vocabulary (#15) as completed

In GitLab by @laureD19 on Aug 5, 2021, 12:07

are the different types of vocabularies, as defined by @KlausIllmayer in the comment above (Internal, open vocabularies; External, closed vocabularies; Internal/external, controlled vocabularies) marked/specified somewhere in the system?

Would you agree with the following?

Internal: keywords, standard
External: iso-639-3, TADIRaH, EOSC vocabulary, software-license, eosc-life-cycle-status, iana-media-type, discipline, EOSC categories
Internal/external: invocation-types, technical-readiness-level vocabulary, audience

(Still need to be classified: publication-type, service-type)

notify also @egray523

In GitLab by @KlausIllmayer on Aug 5, 2021, 12:26

in which way should we define these different types? i tried to implement it - but not consistent - in the name of the vocabulary, like eosc-life-cycle-status which tells us, that the vocabulary is from eosc (but i missed it for discipline). the only way i see is that we clearly mark it in the correspondending gitlab-issue, implying that for some vocabularies new candidate concepts may be problematic (e.g. do we extent an external vocabulary in the MP thus being not compliant to the original vocabulary anymore). so the general question here is more about the workflow we apply to the different types of vocabularies, e.g. candidate concepts for external vocabularies must be accepted by the external source of the vocabulary (which often may not work very well).

In GitLab by @laureD19 on Aug 5, 2021, 13:56

true. It is more about the workflows. What I am interested in is whether or not users will be able to add candidate concepts for all types of vocabularies. As I understand it, yes for internal vocabs and no for external ones. But what about the internal/external ones?

If we say that candidate concepts can be created/suggested by users for internal/external, what is the difference, in terms of workflow, between the internal/external and the external?

In GitLab by @KlausIllmayer on Aug 5, 2021, 14:11

currently, users are able to apply candidate concepts for all vocabularies. it is up to the curators to agree/disagree on such candidate concepts. for a vocabulary like iso-639-3 in my opinion we should always disagree (or to put it different: we can propose to the responsible iso committee to add a new concept but i think it is out of our scope to add such a concept by us: so it will be disagreed on our side but proposed to iso and maybe they then agree/disagree, but this only time will show). for other vocabularies we need to decide from case to case (especially for shared ones with other SSHOC projects like invocation-type: does it imply, that it should be also added at the training discovery toolkit?).

so it is about the workflow for curators: the decision if you agree/disagree on a candidate concept should be based on the type of the vocabulary and if curators agree to a concept from an external vocabulary, initiate a communication to the original vocabulary curators, if they are willing to add a new concept there. but i guess, there is no one-rule for all, as we sometimes mix stuff.

i think the more important question here is how we communicate back to users the change of state of a candidate concept: both agree and disagree - the later one probably with a reason. we need to think how we can identify who proposed a concept and how we can communicate to this user.

In GitLab by @KlausIllmayer on Aug 5, 2021, 14:15

and to add: agree/disagree is not only based on the vocabulary type, it is also about how the candidate concept convinces curators. is it a concept that maybe is used only once (then probably we disagree) or does it have the potential to be used by many others? is there a similar concept that can be used (even if the user propsing the candidate concept does have another opinion on the similarity).

In GitLab by @KlausIllmayer on Aug 5, 2021, 14:33

marked the checklist item discipline using subset of ÖFOS classification (#8 ) -> as completed

In GitLab by @KlausIllmayer on Aug 5, 2021, 14:34

marked the checklist item intended-audience using Training Discovery Toolkit and EOSC (#14) -> as completed

In GitLab by @KlausIllmayer on Aug 5, 2021, 14:34

marked the checklist item keyword turning the values we have into concepts (#18 ) as completed

In GitLab by @vronk on Sep 6, 2021, 13:27

proposition for the combined internal/external: have two allowedVocabularies for a dynamic property: one with external source, one for new concepts not available in the external source.

In GitLab by @laureD19 on Oct 18, 2021, 18:37

refined list/ last version of the vocabulary list in D7.2 - https://docs.google.com/document/d/1xB_Bq4kAqPLjSE1eJ_okln7BpqramqLl/edit#bookmark=id.28jydyoegu1y -

an additional question: I don't see the added-value to allow users to add concepts for closed vocabularies (such as TADIRAH) for example. Would you agree that we only need the candidate concept option for the open vocabularies? If yes, this needs to be changed also in the edit forms (cf. https://gitlab.gwdg.de/sshoc/sshoc-marketplace-frontend/-/issues/73 )

In GitLab by @vronk on Oct 19, 2021, 10:13

indeed, to disallow adding concepts for closed vocabularies would be more consistent. However this means we would need to introduce some flag in the system for this.

In GitLab by @vronk on Nov 28, 2021, 17:42

marked this issue as related to #18

In GitLab by @vronk on Nov 28, 2021, 17:43

unassigned @vronk

In GitLab by @vronk on Nov 28, 2021, 17:44

Set this on Final release for now, but not quite sure can we close this (soon) or is it rather Ongoing curation

In GitLab by @vronk on Nov 28, 2021, 17:47

marked this issue as related to #13

In GitLab by @vronk on Nov 28, 2021, 17:58

marked this issue as related to sshoc-marketplace-backend#125

In GitLab by @vronk on Jan 31, 2022, 16:06

marked the checklist item service-type vocabulary to be determined (#13) -> as completed

In GitLab by @vronk on Jan 31, 2022, 16:07

@KlausIllmayer, I hope you won't disagree with closing this. BUt feel free to reopen if really necessary.

SSHOC / vocabularies

Vocabularies umbrella issue #17