Request To Review: Analyzed Phenotypes Upload

[reynoldtan]

Requesting to review first release of Analyzed Phenotypes Data Loader

[reynoldtan]

Analyzed Phenotype Data Uploader/Loader. This module will handle submission of analyzed phenotypic data. Like rawphenotypes module, this module supports drag and drop functionality and automated file validation. The validation process is broken into 4 stages namely, upload, validate, describe and save. A data viewer using bean chart is provided to show a visual summary of values.

Upload: In this stage data collector is instructed to provide an experiment/project title, a genus and a data file in tab delimited format.

Validate. In this stage, data in rows and columns are validated for empty values, type mistach and values inconsistent with what is expected.

Describe. All traits must be described by data collector. In this stage user is requested to define or describe all traits found in the data file.

Save. Finally, file, data and all will be saved into the system.

Requirements

1. A Tripal clone/site.
2. Drag and Drop module.
3. Admin menu module (creates a nice drop down menu)

To Test:

1. Create a test record for the following terms: A. A test record in chado.db (eg. test_db) - this tells the system which database new cvterms belong. B. A test vocabulary (cv) in chado.cv (eg. test_cv) - this tells the system the cv_id of new cvterms.

2. Install the module.

3. Update system settings. click Configure link next to the module in module list or tripal/extensions/analyzedphenotypes/Settings.

   Crop ontology tab - match a genus to a set of ontology terms suggested in crop ontology field when describing a trait in stage 3. Default crop ontology term is used when nothing matches to given a trait name.

   Database and cv tab: As described above. Other terms in this page include:

   • genus - in taxonomic_rank cv
   • related - in sysnonym_type cv
   • a term to describe the measurements were taken by eye (used as assay_id, not sure what) taxonomy - in tripal_phylogeny cv

   Plant Properties tab: Locate cvterm for the following terms:

   • Year - in tripal_pub cv
   • Location - in nd_geolocation_property
   • Replicate - in local cv
   • Data Collector - no sure

4. Upload a file. tripal/extensions/analyzedphenotypes/Upload Analyzed Phenotypic Data

Validation Triggers In stage 1:

• No project selected
• Project entered not in chado.projects table
• No genus selected
• When project has a genus set, the two must match, otherwise, it will be a project and genus do not match error.
• File is not readable
• File is neither tsv nor txt
• One or more of the expected columns is missing Expected columns (case sensitive): Trait Name, Germplasm Accession, Germplasm Name, Year, Location, Replicate, Value and Data Collector
• When system is set to not allow new terms to be added, a warning in page indicating this setup
  and when data file contains a new trait (assumes user attempts to add new trait)

In stage 2:

• Empty values - all columns must have a value.
• Columns have data type and value does not match type:
  Trait name : must be in this format Trait name (unit) Germplasm Accession : alphanumeric Germplasm Name : alphanumeric Year : Four digit year from 1900 to current year Location : text Replicate : number Value : Depends on the (unit) part of the trait name. Data Collector : text
• Unit in trait name and value do not match. Known units: cm - number g - number kg - number days - number no zero y/n/? - Yes (y), No (n) or Unsure (?) count - number colour - text comments - text scale 1-5 - 1,2,3,4 and 5 scale 1-10 - 1,2,3,4,5,6,7,8,9,10 mn - millimolar number percent - number average count - number
• Germplasm accession and germplasm name do not exists.
• Column combination Trait Name + Germplasm Accession + Year + Location + Replicate matches a record in the system

In stage 3:

• Field description is empty
• Crop ontology is empty or unrecognized value entered
• File 1 of 2 is empty or file type is not jpg

Data Summary Chart in home/phenotypes (summary page) and home/phenotypes/trait-distribution (generate trait distribution chart).

Notes

1. Commented out trait unit, plant ontology, trait method and trait scale - have not figured out a way to store these details.
2. Plant property cvterms are not ideal at the moment.
3. How to relate scale to a trait - have not figured out a way to link these information.

[laceysanderson]

Is this still waiting for review? and is it at a point for me to do so?

[laceysanderson]

I've reviewed the installation & configuration process:

• I've made a few simple changes to the module info & installation: f0b47b1509fc2d86a265afa6a1bb14bdae0132c8, a3421dbcaff94cd951315b519647d344ffe0d108, 5a7da6b528d658e86346b6de0617ff2930235098
• Made a small change to the admin pages: c2dde23444c7923e210dc3748712e6b8c8ef5a2b
• Suggested changes for the configuration form: #9
• Found an oversight: separate trait vocabulary per genus #10
• Also, we need better permissions handling #11. Note: it is never good to set the permission for a menu item to TRUE ;-)

I'm holding off on further review until we get these items cleaned up.

[laceysanderson]

Also, this likely requires an update to work with the new system variables.