SOI_harmonization

The data harmonization repository for the ESIP Soil Ontology and Informatics cluster.

Roadmap

1) Describe current soil data products for a set of key measurement types (see Issues) 2) Link measurement names to existing ontologies or extend those ontologies 3) Develop a hierarchical understanding of the methodologies (new ontologies/knowedge maps?) 4) Work up a work flow that combines proposed data descriptions and a generalized script to generate an integrated data product

Ways to contribute:

1) Identify soil data products for evaluation via https://github.com/ESIPFed/SOI_harmonization/issues/2 2) Describe how layer location (geolocation, depth, and sampling time) are described in the data product https://github.com/ESIPFed/SOI_harmonization/issues 3) Attend SOI Cluster meetings

Propsed data descriptors

The purpose of these proposed data descriptors are to provide a sufficent description of datasets that will enable a general harmonizaiton script to merge datasets into a common database. We are assuming the data is presented as a relational database. Take from https://github.com/ESIPFed/soil_data_model_survey/tree/master/data

1) data_structure: Description of the columns in a database

• data_product: ID for the dataset being described. Free text, should be a unique ID
• data_table: ID for the data table being described. Free text, should be a unique ID.
• data_column: Column name. Free text, should be a unique ID.
• data_type: Type of the data in that column. Control vocabulary. Suggested vales:
  • value_number: a numerical value
  • value_factor: a control vocabulary value
  • value_string: a character value or free text
  • unit: a unit, generally associated with a value column
  • method: a method description, generally associated with a value column
  • id: a unique identifier
  • note: a note, generally associated with a value column
  • sigma: standard deviation/sigma uncertainty

2) data_meta: Descriptions of the metadata that applies to the entire table

• data_product: ID for the dataset being described. Free text, should be a unique ID
• data_table: ID for the data table being described. Free text, should be a unique ID. Could be NA, suggesting meta applies to entire product
• data_column: Column name. Free text, should be a unique ID. Could be NA, suggesting meta applies to entire table
• data_type: Type of the data in that column. Control vocabulary. Suggested vales:
  • value_number: a numerical value
  • value_factor: a control vocabulary value
  • value_string: a character value or free text
  • unit: a unit, generally associated with a value column
  • method: a method description, generally associated with a value column
  • id: a unique identifier
  • note: a note, generally associated with a value column
  • sigma: standard deviation/sigma uncertainty
• entry: assigned value to that property for the product/table/column of interest

3) thesaurus: Description mapping provided variable to a common vocabulary

• variable_location: Where is the variable described in the dataset? Either a column_name or variable_name to cover wide and long tables
• data_product: ID for the dataset being described. Free text, should be a unique ID
• data_table: ID for the data table being described. Free text, should be a unique ID. Could be NA, suggesting meta applies to entire product
• provided_variable: variable name from the original dataset, often column header matching data_column in data_meta
• variable: common vocabulary that is being mapped to this should reference some external URI