⚠️ DEPRECATED ⚠️

This fork has been deprecated.

• For issues regarding DataHarmonizer itself see: https://github.com/cidgoh/DataHarmonizer
• For issues regarding the integration of DataHarmonizer into the NMDC Submission Portal see: https://github.com/microbiomedata/nmdc-server
• For issues regarding the schema used by DataHarmonizer in the NMDC Submission Portal see: https://github.com/microbiomedata/submission-schema

This repo is a combination of

• cidgoh/DataHarmonizer
• the MIxS and NMDC schemas (as submodules)
• code that can convert between spreadsheets, one or more LinkML files, and a DataHarmonizer data.tsv file
  • cidgoh/DataHarmonizer may be adopting its own LinkML conversion mechanism, but moving away from the data.tsv format

It's possible to regenerate a DH interface based on MIxS soil and NMDC biosample by

• obtaining a Google client secrets JSON file that authorizes code in this repo to read Google Sheets. Place in local/client_secret.apps.googleusercontent.com.json
  • additional interactive validation may be required the first time any code from this repo is executed
  • more documentation to come
• cloning this repo
• installing the python poetry tool
• installing the poetry environment
• configuring GH pages for this repo (main branch, docs directory)
• make all

Want to run some tests? Try poetry run pytest -rsvv tests/

original cidgoh/DataHarmonizer DataHarmonizer README

A standardized spreadsheet editor and validator that can be run offline and locally, and which includes templates for SARS-CoV-2 sampling data.

Watch Rhiannon Cameron and Damion Dooley describe this application on YouTube at the Canadian Research Software Conference (CRSC2021).

Installation

Download the zipped source code from the latest release at: https://github.com/Public-Health-Bioinformatics/DataHarmonizer/releases

Extract the zipped file.

To run the application, navigate to the extracted folder and open main.html.

Select Template

The default template loaded is the "CanCOGeN Covid-19" template. To change the spreadsheet template, select the white text box to the right of Template, it always contains the name of template currently active, or navigated to File followed by Change Template. An in-app window will appear that allows you to select from the available templates in the drop-down menu. After selecting the desired template, click Open to activate the template.

A second way to access templates directly, rather than by the hard-coded menu system, is to specify the DataHarmonizer template subfolder via a "template" URL parameter. This enables development and use of customized templates, or new ones, that DH doesn't have programmed in menu.

For example, http://genepio.org/DataHarmonizer/main.html?template=gisaid accesses the /template/gsiaid/ subfolder's template directly.

See more on the Wiki DataHarmonizer templates page.

Usage

You can edit the cells manually, or upload xlsx, xls, tsv and csv files via File > Open. You can also save the spreadsheet's contents to your local hard-drive in the aforementioned formats, or File > Export your data as an .xls document formatted for submission a specified portal, database, or repository.

Click the Validate button to validate your spreadsheet's values against a standardized vocabulary. You can then browse through the errors using the Next Error button. Missing value are indicated in dark red, while incorrect values are light red.

Double click any column headers for information on the template's vocabulary.

You can quickly navigate to a column by selecting Settings > Jump to.... An in-app window will appear, select the desired column header from the drop-down list or begin typing it's name to narrow down the list options. Selecting the column header from the drop down list will immediately relocate you to that column on the spreadsheet.

You can also automatically fill a column with a specified value, but only in rows with corresponding values in the first sample ID column. To use this feature select Settings > Fill column.... Select the desired column header from the drop-down list or begin typing it's name to narrow down the list options, then specify the value to fill with and click Ok to apply.

For more information on available application features, select the Help button followed by Getting Started from within the DataHarmonizer application.

Additional Information

For more information about the DataHarmonizer, it's templates, and how to use them, check out the DataHarmonizer Wiki.

Support

If you have any ideas for improving the application, or have encountered any problems running the application, please open an issue for discussion.

Roadmap

This project is currently in the beta phase, with new features being added occasionally.

Acknowledgement

• Handsontable was used to build the grid. DataHarmonizer is configured to reference the "non-commercial-and-evaluation" handsontable license "for purposes not intended toward monetary compensation such as, but not limited to, teaching, academic research, evaluation, testing and experimentation"; if this application is used for commercial purposes, this should be revised as per https://handsontable.com/docs/license-key/
• SheetJS was used to open and save local files. The community edition was used under the Apache 2.0 license.

License

DataHarmonizer javascript, python and other code not mentioned in the Acknowledgement above is covered by the MIT license.