UC9 - Keyword semantic data search

[kitchenprinzessin3880]

Keyword semantic data search

Data discovery based on keywords that come from a controlled vocabulary

Corresponding user stories

• data manager or scientist - nitrogen in biomass and fertilization - model /AnaEE02
• Scientist - assess data - to generalize trait-environment-relationships in phytoplankton communities

Requirements identified in the spreadsheet (and open for discussion)

• [ ] Requires concepts that have persistent resolvable URIs
• [x] Requires the terminology be hosted by a trustworthy semantic repository
• [ ] Requires terminologies based on simple atomic terms (e.g. used to make more complex compound terms)
• [ ] Requires (agreement of) top-level, domain-independent categorization scheme/ontology
• [ ] Requires that the terminology includes semantic relations between concepts
• [ ] Requires mappings between terminologies
• [ ] Requires that the provenance and status metadata of mappings between concepts be accessible.
• [x] Requires coarse-grained terminologies for observable properties
• [x] Requires fine-grained terminologies for observable properties
• [x] Requires a long-term commitment to the terminology content governance
• [x] Requires a community-endorsed terminology
• [x] Requires the terminology to be discoverable (not just technically but also e.g. listed in relevant community portals or discoverable via search engines)
• [ ] Requires that the terminology supports multilingual terms
• [ ] Requires common minimum metadata describing the terminology
• [ ] Requires definitions of the individual terms within a terminology
• [ ] Requires that any underlying model used by the terminology be described and accessible
• [ ] Requires the provenance properties of individual terms to be documented in a standardized way
• [ ] Requires examples of usage of the terminology to be available and documented
• [ ] Requires the terminology to be accessible programmatically
• [ ] Requires deprecated term status and deprecation policy
• [ ] Requires tools supporting annotation using the terminologies
• [ ] Requires tools validating the applied terms against terminologies
• [ ] Requires tools for mapping
• [ ] Requires a mean to manage alternative notations

last updated: 2020-09-21

[kitchenprinzessin3880]

Assumption: A data provider applies the harvested ontological terms as part of full-text search on the data portal. Requirements: R1. Term name, alternative name, vernacular name (species) R2. Term acronym/abbreviation (chemincal formula) R3. Term description R4. Term relations (e.g., we use skos-based relations), synonym

[SirkoS]

(my additional requirements, as usual, a little more into the details)

entities for ...

• method or process to derive the measurement
• kind of quantity
• compounds of the kind of quantity
  • i lack a better word here. basically it's meant to support the "nitrogen" in "concentration (kind of quantity) of nitrogen (compound)"
• traits
  • not sure, if this is a subclass of kind of quantity. currently tending to "no"
• object for which the measurement was made, e.g., some plant
• datasets to combine multiple measurements

[mariutzica]

@SirkoS These are good additions. Perhaps clarify as follows: the object and compound line items make up the system being observed, while the kind of quantity is the quantitative property being measured/modeled/computed. A requirement should be to identify all elements of the observed system minimally necessary to understand the quantity recorded—for this case the concentration is measured, which is defined as the amount of A in B, so both A and B must be specified. Perhaps also add that a valid quantity should be accompanied by dimensions and a unambiguous type (more of a requirement for resource alignment) ... so, for example, specify mass concentration, M/L^3, vs volume concentration L^3/L^3.

[SirkoS]

I agree that those are important, but I feel we can easily get lost in details. So there should be extension points where we do not care about the details and just acknowledge that there is something important. From my perspective, the dimensions are such a think. I don't think it's important to the observation itself what is beyond the point of "kind of quantity" like dimensions or dimensional vectors. So for me, this would be something that is covered elsewhere, but not within our activities.

[kitchenprinzessin3880]

here are ontological terms observation 'types' used in pangaea data search

1. methods, instruments
2. observed properties/quantities
3. features (including specifies)
4. we dont use vocabularies for unit as represent them using ucum convention

[gwemon]

I have transferred the requirements listed here into the list of requirements tab of the spreadsheet so that we include them for the discussion tomorrow. I have condensed Sirko's suggestions into 4 requirements:

• Terms to describe the quantity or property measured/observed/modelled
• Terms to describe the method or process to derive the measurement
• Terms to describe the system/object observed (e.g. chemical substance, plant, species, physical object)
• Terms to describe traits

Not sure if "trait" deserves its own requirement neither. I think it may be part of the description of the system/object observed, in this case a biological object.

[mariutzica]

@gwemon I am not 100% sure, but I think trait came from US 16 to refer to ‘morpho-functional’ properties such as ‘shape’. If so, I think this would fall under the property category.

[gwemon]

thanks @mariutzica. There are also cases when the observation is about an organism of a particular trait (this happens a lot in plankton sample analysis where the organism is not fully identified but it is described using characteristics of shape, colour, size categories). Also, organisms can be counted according to their biological sex category or life stage. In these cases, the trait would be needed as part of the observed object description. So it might be safer to keep traits separate indeed at least for the time being?

[mariutzica]

@gwemon I see, thanks for explaining! So 'traits' in this sense means the terms that indicate fixed values of properties (whether intrinsic or not) for the duration of a measurement/observation. In SVO we have a special class for this purpose called Attribute which is used to represent Property-Value pairs (i.e., every Attribute is associated with a Property) for values that are not presented in the data. We found this helpful because sometimes data is condensed or unpacked by these attributes (e.g., for one dataset particle size may be part of the terminology while in another it may be provided its own column in the dataset). Trait in this sense definitely deserves its own bullet point.

[gwemon]

Notes transferred from Maria's document for UC9 keyword semantic data search (data discovery based on keywords that come from a controlled vocabulary):

• Notes:

  • keyword semantic data search is greatly aided by the inclusion of synonyms (skos:altLabel)
  • main I’ve had with this kind of system is NASA’s GCMD Keywords

• Requires concepts that have persistent resolvable URIs

  • not required

• Requires terminologies based on simple atomic terms

  • not required, but the more atomic the structure, the better the semantic search results

• ADDED - Requires agreement of top-level, domain-independent categorization scheme/ontology

  • not required

• Requires that the relationships be trusted

  • not required

• Requires terminologies with coarse/fine granularity

  • not required
  • the finer the granularity, the better the search results

• Requires a long-term commitment governance setup

  • required in the maintenance of the controlled vocabulary
  • the way the CV is populated directly affects semantic search capabilities

• Requires an active community supporting the terminology

  • think so, but not sure
  • community can aid in providing alternate labels/synonyms

• Requires reliable technical infrastructure

  • yes (?)

• Requires input from domain experts

  • required in the maintenance of the controlled vocabulary

• Requires that the terminology be part of federated community specific and/or cross-domain portals

  • don’t think so

• Requires that the terminology supports multilingual terms

  • ideal, but not necessary

• Requires multilingual editorial team or multilingual community effort

  • ideal, but not necessary

• Requires terminologies published as linked data capabilities

  • very helpful, but not required

• Requires the terminology to use a common minimum metadata schema to describe semantic artefacts and their content

  • very helpful, possibly required (?)

• Requires mappings between terminologies

  • not required

[gwemon]

Summary of Maria's analysis for this UC:

• Requires terminologies with coarse granularity
• Requires a long-term commitment governance setup
• Requires an active community supporting the terminology
• Requires a reliable technical infrastructure
• Requires input from domain experts

[gwemon]

Requirements identified in the spreadsheet (and open for discussion)

• [ ] Requires concepts that have persistent resolvable URIs
• [ ] Requires terminologies managed in a semantic repository
• [ ] Requires terminologies based on simple atomic terms
• [ ] Requires (agreement of) top-level, domain-independent categorization scheme/ontology
• [ ] Requires semantic relations between concepts
• [ ] Requires mappings between terminologies
• [ ] Requires that the provenance of mapping properties be trusted
• [x] Requires terminologies with coarse granularity
• [ ] Requires terminologies with fine granularity
• [x] Requires a long-term commitment governance setup
• [x] Requires an active community supporting the terminology
• [x] Requires a reliable technical infrastructure
• [x] Requires input from domain experts
• [ ] Requires that the terminology be part of federated community specific and/or cross-domain portals
• [ ] Requires that the terminology supports multilingual terms
• [ ] Requires multilingual editorial team or multilingual community effort
• [ ] Requires terminologies published as Linked Data
• [ ] Requires the terminology to use a common minimum metadata schema to describe semantic artefacts and their content
• [ ] Requires definitions of the concepts within a terminology
• [ ] Requires used data models to be explicit and accessible
• [ ] Requires the terminology to track the provenance of individual terms

[SirkoS]

^{[automated message]} Updated top entry of this issue on 2020-07-11

[SirkoS]

• Requires a long-term commitment governance setup
• Requires input from domain experts

Why are those needed?

• Requires that the terminology supports multilingual terms
• Requires alternative syntax (like alternative notation e.g. chemical formula, abbreviations)
• Requires synonym management

Not mandatory, but helpful.

Requires definitions of the concepts within a terminology

Maybe helpful for disambiguating search queries.

[gwemon]

Agree with @SirkoS that the following: Requires a long-term commitment governance setup Requires input from domain experts may not be needed in order to support keyword semantic data search. Could somebody provide explanations supporting these?

[gwemon]

Also agree about @SirkoS selection of useful others for this use case. But because they are not essential to the use case, I would pefer to leave them out. Does everybody still agree that at this stage we need to limit ourselves to only record "key" requirements, i.e. the requirements without which a use case cannot be addressed?

[mariutzica]

@gwemon i agree — these are not strictly needed for the search step; they are needed for the terminology maintenance/update step

[SirkoS]

^{[automated message]} Updated top entry of this issue on 2020-09-21