HUS001 - Build passport indicators

[juangarc]

Description A user wants to be able to import a set of passport indicators for each accession, with which the users will be able later to create Genebank accessions subsets from filters based on these passport indicators.

Acceptance criteria

1. The file with the data extracted of Genesys must be in JSON Format.

• [x] Stage 1 - – Import Genesys passport data:  To import the passport data since genesys to the tool, the administrator user must activate the option “Import accessions passport data since Genesys” and the system will fetch the passport indicators for each accession such as (Location information, tag if the accession is core, institution information biological status). 
• [x] Stage 2 -- Successful operation: For each stage, if the operation flow is successful the system shows a message indicating it.
• [x] Stage 3 -- Failed operation: For each stage, if the file structure isn't correctly, the system will show an exception with the message: "The file structure isn't correct.".
• [x] Stage 4 -- Generate logs file:  For each stage the system will generates a log file, which is described in the HUS007.

[khadijaAouzal]

I think the passport indicators do not have to be built. We just need to extract them from Genesys.

[khadijaAouzal]

Is the objective of building passport indicators to have all the accessions available in Genesys with their related passport data at design time?
What do you mean by activating the option "Import accessions passport data from Genesys"? Is it for future use, for example for updating the data?
Also, in stages 3 and 4 of the user story, which file are you referring to?

[stevensotelo]

@juangarc

• we need more details about the user story like format file

@khadijaAouzal

• limit of the request at the day (data is huge)
• the download will be by crop

@brayan9487

• we can start with cassava

[stevensotelo]

Kehel said:

• [ ] why sometimes we have "," and sometimes we have "." in the latitude and longitude and some of the numbers are strange!. I know it is an example but we need some quality control once the data is extracted from genesys
• [x] we also need to make the difference when filtering between donor.institute and institute. The focus should be on the institute and its code and not donor

[stevensotelo]

s it for future use, for example for updating the data?

1- The idea is not to have a copy of all genesys databases into the system, but yes all data about passport for all accessions. When I said "all data", I mean that we need data which is the requirements from people (survey) 2- Yes, it is when we need to update data 3- It is to manage execptions in runtime, if you would run into a problem when the script is running, you should create a log and write in this file what happened

[stevensotelo]

@khadijaAouzal

• [x] She sent the genesys data for cassava

• [x] She will add logging statements (R package)

• [x] She will sent a list of package in R in order that @juangarc can add them into docker image

• [x] @juangarc send you the list of columns which will use into the system

• [x] Standarization columns names

Note: Example of how will work the master script:

genesys = extract_passport_data( ["casssava","maize"]) save_into_db(genesys) values = water_indicator(genesys) save_into_db(values)

[khadijaAouzal]

• [x] She will sent a list of package in R in order that @juangarc can add them into docker image

@juangarc This is list: genesysr, jsonlite, and futile.logger.

For the genesysr package, I recommend to install the dev version from gitlab, since it has additional features:

devtools::install_git('https://gitlab.croptrust.org/genesys-pgr/genesysr', ref = '2-welcome')

Eventually, you may need to install the devtools package if not installed.

[khadijaAouzal]

@stevensotelo, @juangarc To authenticate to Genesys, I used user authentication which requires user interaction to enter the authorization code. We can modify this later by using the client authentication.

Refer to https://gitlab.croptrust.org/genesys-pgr/genesysr for more details.

And I think it'd be better to include the authentication in the master script, or in a higher level of the app.

[stevensotelo]

Issues:

• [x] Ask to @jramirezv and zack if we should save all records, even if they don't have coordinates

[juangarc]

@stevensotelo, @juangarc To authenticate to Genesys, I used user authentication which requires user interaction to enter the authorization code. We can modify this later by using the client authentication.

Refer to https://gitlab.croptrust.org/genesys-pgr/genesysr for more details.

And I think it'd be better to include the authentication in the master script, or in a higher level of the app.

Yes @khadijaAouzal , I think the same. I'll review this documentation and I'll implement the Genesys client auth from our app

• [ ] add Genesys client auth from app when the users need to import new data from Genesys

[juangarc]

These are the columns:

• accessionName
• accessionNumber
• countryOfOrigin.name
• countryOfOrigin.regionName
• countryOfOrigin.code3
• cropName
• doi
• elevation
• latitude
• longitude
• id
• instituteAcronym
• instituteCode
• instituteName
• available
• coll.collDate
• taxonomy.genus
• taxonomy.spAuthor
• taxonomy.species
• taxonomy.subtAuthor
• taxonomy.subtaxa
• taxonomy.taxonName
• mls
• sampStat