BrAPI to ISA

Code to pull data from BrAPI endpoints and create an ISA representation of the experiments in a MIAPPE compliant way. This script is part of the Data Validation implementation study of ELIXIR.

Validation scheme

The mapping from ISA to MIAPPE is fully documented here: https://github.com/MIAPPE/ISA-Tab-for-plant-phenotyping/blob/v1.1/README.md

Dependencies

python 3.7 +
following python modules:
- isatools
- requests
- pycountry-convert
- cachetools
- markupsafe

NOTE: For better MIAPPE compliance, make sure to have isatools v0.12.0 + installed.

Installation

Allows to use this program as a python pip package.

Install the package with:

pip install git+ssh://git@github.com:elixir-europe/plant-brapi-to-isa.git@master#egg=brapi2isa

or clone this repository and exicute this in the root of the directory:

pip install .

Usage

brapi2isa [options…]

or exicuting in the cloned directory:

python brapi_to_isa.py [options…]

Mandatory options:

-e, --endpoint A BrAPI server endpoint
-t, --trials Comma separated list of trial Ids. Use 'all' to get all trials\ or
-s, --studies Comma separated list of study Ids

Optional:

-J, --json flag to deactivate json dump
-V, --validator flag to deactivate validation
-F, --flatten flag to generate a flattened data file based on observationTimStamp

Input

A valid BrAPI endpoint with following GET calls implemented:

/trials
/trials/{trialDbId}
/studies/{studyDbId}
/studies/{studyDbId}/observationunits or /phenotypes-search
/studies/{studyDbId}/observationvariables
/studies/{studyDbId}/germplasm and possibly
/germplasm/{germplasmDbId}

Test your endpoints here for BrAPI compliance.

BrAPI to isa is tested and optimized for BrAPI version 1.0, 1.1, 1.2 and 1.3. NOTE: BrAPI 2.0 is NOT yet supported

Output

Meta-data in ISA-tab
- 1 investigation file (i_investigation.txt)
- 1 study file / study (s_*.txt)
- 1 assay file / study / observation level (a_*.txt)
Meta-data in 1 ISA-JSON file (*.json)
1 Trait definition file / study (t_*.txt)
1 Data file in tabular format / observation level (d_*.txt)
1 Validation log file (*_validation_log.json)

Output will be put into a subfolder /outputdir.

Unavailable values

BrAPI v1.3 and earlier does not support all the necessary attributes that are needed for MIAPPE compliance. These fields will be filled in with "NA in BrAPI". Be aware that these fields are not detected by the validator since they are filled in with a string.

Values that are supported by BrAPI but are not implemented in the given endpoint, will be filled in with "NA in endpoint".

Validation

The dumped ISA-tab files are automatically validated by the ISA-API using the MIAPPE configuration files. This to ensure MIAPPE compliance. Check the generated validation_log.json file for more details.

External resources

To determine the NCBI taxon-id following link is used: https://www.ebi.ac.uk/ena/taxonomy/rest/any-name/

To fetch all the ontologies and there corresponding information, following link is used: http://www.obofoundry.org/registry/ontologies.jsonld

Tested Examples

python brapi_to_isa.py -e https://urgi.versailles.inra.fr/faidare/brapi/v1/ -s dXJuOlVSR0kvc3R1ZHkvUklHVzE=
python brapi_to_isa.py -e https://urgi.versailles.inra.fr/faidare/brapi/v1/ -t aHR0cDovL2R4LmRvaS5vcmcvMTAuMTE4Ni8xNDcxLTIyMjktMTItMTcz
python brapi_to_isa.py -e https://urgi.versailles.inra.fr/faidare/brapi/v1/ -t dXJuOlVSR0kvdHJpYWwvNw==
python brapi_to_isa.py -e https://pippa.psb.ugent.be/BrAPIPPA/brapi/v1/ -t all
python brapi_to_isa.py -e https://www.eu-sol.wur.nl/webapi/tomato/brapi/v1/ -t 2
python brapi_to_isa.py -e https://brapi.biodata.pt/brapi/v1/ -t 2

Docker

This setup is to be used with Docker for easy dependency requirements.

Running BrAPI to ISA in a container without test containers:

docker build -t brapi2isa -f Dockerfile .

docker run -it -v <absolutepath>/outputdir:/outputdir brapi2isa -t <your trial DbId> -e <your endpoint>

Usage for your own endpoint:

docker-compose build && docker-compose run BrAPI2ISA -t <your trial DbId> -e <your endpoint>

Usage for testing:

docker-compose build && docker-compose run <>

Where <> can be following values:

test_trial
test_study
test_pippa