Add concept types endpoint

nichtich commented 1 year ago

Some vocabularies have their concepts grouped in types of concepts:

Classifications such as RVK and DDC have normal classes and number spans. Compound classes may be another independent type of classes
Thesauri often have their concepts grouped in a (or multiple?) classifications.
- e.g. GND has entity types Person, Corporate Body, Work... with more detailed subdivision
- STW has a subject classification of concepts.
Note that both are currently modeled differently: In GND entity types are "types" in the sens of this issue, in STW the subject classification is part of the concepts but classes have an additional type "zbwext:Thsys" to distinguish from normal concepts.
RDF Ontologies have classes and properties as two basic types of entities

Some open questions:

types are vocabularies of their own, should they be included at other endpoints as well?
this should be aligned with cocoda-sdk and with other terminology services such as Skosmos and Ontology Lookup Service (OLS)
Define /types endpoint as already being provided by in DANTE API.

nichtich commented 1 year ago

GND

GND Ontology has seven types (not sure about one of them):

Every concept must have one of these types (mandatory type). GND ontology has more fine-grained classification of concepts but "types" should be limited to basic types.

PICA Format for authority recordings, subsuming GND internal format has several types but they can be stripped down to the same (except Family):

Corporate Body
Subject heading
Conference or Event
Place or geographic name
Person
Work

STW

There is the additional type zbwext:Thsys for classes of STW systematics. Normal concepts used for indexing don't have this type, so there is only one optional type.

RDF Ontologies

Two disjoint mandatory types: classes and properties.

Skosmos API

See https://api.finto.fi/doc/#!/Global32methods/get_types and https://api.finto.fi/doc/#!/Vocabulary45specific32methods/get_vocid_types. For instance https://api.finto.fi/rest/v1/yso/types?lang=en:

[
  {
    "uri": "http://www.w3.org/2004/02/skos/core#Concept",
    "label": "Concept"
  },
  {
    "uri": "http://www.w3.org/2004/02/skos/core#Collection",
    "label": "Collection"
  },
  {
    "uri": "http://purl.org/iso25964/skos-thes#ConceptGroup",
    "label": "Concept group"
  },
  {
    "uri": "http://purl.org/iso25964/skos-thes#ThesaurusArray",
    "label": "Array of sibling concepts"
  },
  {
    "uri": "http://www.yso.fi/onto/yso-meta/Concept",
    "label": "General concept",
    "superclass": "http://www.w3.org/2004/02/skos/core#Concept"
  },
  {
    "uri": "http://www.yso.fi/onto/yso-meta/Individual",
    "label": "Individual concept",
    "superclass": "http://www.w3.org/2004/02/skos/core#Concept"
  },
  {
    "uri": "http://www.yso.fi/onto/yso-meta/Hierarchy",
    "label": "Hierarchical concept",
    "superclass": "http://www.w3.org/2004/02/skos/core#Concept"
  }
]

Only types with "superclass": "http://www.w3.org/2004/02/skos/core#Concept" seem to be types in the sense of this issue.

nichtich commented 1 year ago

There should be API endpoint GET /voc/concepts/types with optional parameter uri to get concept types (distinct values in field type of all concepts, excluding the default skos:Concept), optionally filtered to a vocabulary.

List of concept types (distinct values in field type of all vocabularies, excluding the default skos:ConceptScheme) could be returned at GET /voc/types.

Mapping types (by now hardcoded list) could be returned at GET /mappings/types but this requires further discussion.

nichtich commented 1 year ago

To avoid sub-sub-level endpoint URLs the API endpoint should be /voc/concept-types. Specification and documentation for README.md as following:

GET /voc/concept-types

Lists concept types of a concept scheme. A concept type is an additional type URI in JSKOS field type of a concept. The first and mandatory value http://www.w3.org/2004/02/skos/core#Concept does not count as concept type and is not included in the result list. If concept scheme field types exist, its values is used. Otherwise the list of concept types is determined by inspecting all concepts of the concept scheme.

URL Params

uri=[uri] URI for a concept scheme. If missing, concept types of all concept schemes are returned.

download=[type] returns the whole result as a download (available types are json and ndjson), ignores limit and offset
Success Response

JSON array of [JSKOS Concepts]

Sample Call

curl https://coli-conc.gbv.de/api/voc/concept-types?uri=http://dewey.info/scheme/edition/e23/

stefandesu commented 1 year ago

After the recent changes in #193, should this maybe be adjusted to /concepts/types?

nichtich commented 1 year ago

Yes, it should better be /concepts/types with parameter voc=[uri] (not sure whether to make the parameter optional to return all types of all schemes by default).

stefandesu commented 1 year ago

I feel like the list of all types of all schemes doesn't make much sense.

nichtich commented 1 year ago

I feel like the list of all types of all schemes doesn't make much sense.

Then it should better be /voc/types just like /voc/concepts. Here is second try to formulate specification and documentation for README.md:

GET /voc/types

Lists concept types of a concept scheme. A concept type is an additional type URI in JSKOS field type of concepts in a given concept scheme. The first and mandatory value http://www.w3.org/2004/02/skos/core#Concept does not count as concept type and is not included in the result list.

URL Params

uri=[uri] URI for a concept scheme (mandatory)
Success Response

JSON array of JSKOS Items

Sample Call (sorry wrong URL, fixed it)

curl https://coli-conc.gbv.de/api/voc/types?uri=http://bartoc.org/en/node/18785

Additional ferature: #210

Open question: how this relates to concept scheme field types.

stefandesu commented 1 year ago

Did you mean to use a different URI for the Sample Call? It's not a concept scheme URI. Otherwise I'm fine with that.

Open question: how this relates to concept scheme field types.

I would think they offer the exact same data. The question is whether we actually include the full types list when requesting concept scheme data, as it might inflate the data if many schemes are requested. Maybe it's kind of like narrower and ancestors for concepts: If types exist, it will return types: [null] for the scheme, and then the full list of types can be requested with the /voc/types endpoint.

Another question: Where is the types data located and how is it imported?

nichtich commented 1 year ago

Did you mean to use a different URI for the Sample Call?

Yes, sorry I've fixed it!

About types field I agree: either include the same data as returned by /voc/types or [ null ] or omit the field if status is unknown.

Another question: Where is the types data located and how is it imported?

There are two sources of type URIs:

1.) explicitly list types in field type of a concept scheme, when it is imported/updated: intended list of types 2.) all distinct type URIs found in concepts of a concept scheme: actual list of types

The full list of types could be

a) just 2. so field concept schemes field types is ignored on importing the scheme b) the union of both.

I tend to b): when a concept is added to a scheme, check whether the concept types are already listed in the scheme's type list and add the type URI if it was not present. When a concept is deleted, don't update types.

Additional information about types could come from /data endpoint: by default a type is only known with a bare uri but once there is a Concept with this URI or it's a known URI (#209), return its additional information.

gbv / jskos-server