UC6 - Data annotation

[mabablue]

Data annotation

Manual process for annotation of column headers/fields and streams. Could be done in real- or near-real time when the data are generated and subsequently transformed or in delayed mode.

Corresponding user stories

• Observer- Annotate observation in situ - interoperable descriptions from start
• Scientist - Create standardized descriptions of new particle formation events - Interoperable and reusable data
• Data Manager - ObservableProperty with parameters - create identifiers for property+parameters
• Data manager - interchange between BODC PUV P01 and CF standard names - deliver SeaDataNet CF-NetCDF
• Data manager - improve semantic richness of coastal research metadata - better data interoperability and reusability

Requirements identified in the spreadsheet (and open for discussion)

• [ ] Requires concepts that have persistent resolvable URIs
• [ ] Requires the terminology be hosted by a trustworthy semantic repository
• [ ] Requires terminologies based on simple atomic terms (e.g. used to make more complex compound terms)
• [ ] Requires (agreement of) top-level, domain-independent categorization scheme/ontology
• [ ] Requires that the terminology includes semantic relations between concepts
• [ ] Requires mappings between terminologies
• [ ] Requires that the provenance and status metadata of mappings between concepts be accessible.
• [ ] Requires coarse-grained terminologies for observable properties
• [ ] Requires fine-grained terminologies for observable properties
• [x] Requires a long-term commitment to the terminology content governance
• [x] Requires a community-endorsed terminology
• [ ] Requires the terminology to be discoverable (not just technically but also e.g. listed in relevant community portals or discoverable via search engines)
• [ ] Requires that the terminology supports multilingual terms
• [ ] Requires common minimum metadata describing the terminology
• [ ] Requires definitions of the individual terms within a terminology
• [ ] Requires that any underlying model used by the terminology be described and accessible
• [ ] Requires the provenance properties of individual terms to be documented in a standardized way
• [ ] Requires examples of usage of the terminology to be available and documented
• [ ] Requires the terminology to be accessible programmatically
• [ ] Requires deprecated term status and deprecation policy
• [ ] Requires tools supporting annotation using the terminologies
• [ ] Requires tools validating the applied terms against terminologies
• [ ] Requires tools for mapping
• [ ] Requires a mean to manage alternative notations

last updated: 2020-09-21

[gwemon]

Notes transferred from Maria's document: UC6 data annotation (Manual or automated process for annotation of column headers/fields and streams. Could be done in real- or near-real time when the data are generated and subsequently transformed or in delayed mode.):

• Notes:

  • manual and automated process may have differing requirements
  • m/a notation for requirement — x/x both required, x/ required for manual, /x required for automated

• Requires concepts that have persistent resolvable URIs

  • not required to perform action, but highly desirable, esp for subsequent data discovery

• Requires terminologies based on simple atomic terms

  • not absolutely required, but the more atomic the structure, the better the ability to annotate accurately, so highly recommended

• ADDED - Requires (agreement of) top-level, domain-independent categorization scheme/ontology

  • not required

• Requires that the relationships be trusted

  • not required

• Requires terminologies with coarse/fine granularity

  • not required, but highly desirable
  • the finer the granularity, the better the annotation capability

• Requires a long-term commitment governance setup

  • not required/required - manual annotation is done once; automated annotation is based on some kind of algorithmic approach that will need to be updated/maintained over time

• Requires an active community supporting the terminology

  • required if new terminology needs to be introduced for successful annotation

• Requires reliable technical infrastructure

  • required for automated annotation

• Requires input from domain experts

  • required for manual annotation

• Requires that the terminology be part of federated community specific and/or cross-domain portals

  • not sure

• Requires that the terminology supports multilingual terms

  • ideal, but not necessary

• Requires multilingual editorial team or multilingual community effort

  • ideal, but not necessary

• Requires terminologies published as linked data capabilities

  • very helpful, but not required

• Requires the terminology to use a common minimum metadata schema to describe semantic artefacts and their content

  • very helpful, possibly required (?)

• Requires mappings between terminologies

  • not required

[gwemon]

Summary from Maria's input The following requirements are identified for manual annotation:

• Requires an active community supporting the terminology
• Requires input from domain experts

For automated annotation:

• Requires a long-term commitment governance setup
• Requires a reliable technical infrastructure

[mabablue]

should we make two use cases - manual and automated data annotation?

[SirkoS]

^{[automated message]} Updated top entry of this issue on 2020-07-11

[SirkoS]

^{[automated message]} Updated top entry of this issue on 2020-09-21