Create capability to check/add ITEM_CODE-METHOD_CODE

[annikakj]

In old PetDB there was a controlled vocabulary list which E-Checker read against, thus combinations that were not in this list were flagged as potential errors. DM need this functionality in new Admin/E-checker. I need to have items checked against a controlled list (E-Checker) and capability to add new combinations that will be allowed as well as a place to look up existing combinations (Admin).

Examples of the list:

ITEM_CODE	METHOD_CODE
AB	XRF
AB	CALC
AB	PC
AC	CALC
ACC	CALC
ACC	PC
ACT	PC
ALM	CALC
ALT	PC
AMG	CALC
AMPH	XRF
AMPH	CALC
AMPH	Unknown
AMPH	PC
AN	CALC
ANAL	CALC
AND	CALC
ANDR	CALC
ANH	PC
APA	CALC
APA	PC
AUG	PC
Ag	ICPMS
Ag	ICPAES
Ag	FPHOT
Ag	RNAA
Ag	LA-ICPMS
Ag	Unknown

[klehnert55]

I am not sure we can maintain this list in the future, it is difficult. Especially for Astromaterials.

As our data entry process will change, we will not invest effort at this point to build this functionality. There are more important issues

We will talk when I am back

Sent from my iPhone

On Jan 22, 2019, at 16:05, annikakj notifications@github.com wrote:

In old PetDB there was a controlled vocabulary list which E-Checker read against, thus combinations that were not in this list were flagged as potential errors. DM need this functionality in new Admin/E-checker. I need to have items checked against a controlled list (E-Checker) and capability to add new combinations that will be allowed as well as a place to look up existing combinations (Admin).

Examples of the list:

ITEM_CODE METHOD_CODE AB XRF AB CALC AB PC AC CALC ACC CALC ACC PC ACT PC ALM CALC ALT PC AMG CALC AMPH XRF AMPH CALC AMPH Unknown AMPH PC AN CALC ANAL CALC AND CALC ANDR CALC ANH PC APA CALC APA PC AUG PC Ag ICPMS Ag ICPAES Ag FPHOT Ag RNAA Ag LA-ICPMS Ag Unknown — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub, or mute the thread.

[klehnert55]

And I already see errors in the list. For example AMPH is a mineral and a variable for rock mode and cannot be analyzed by XRF.

The maintenance of this list takes as much effort as it takes to carefully check the methods after data loading

Sent from my iPhone

On Jan 22, 2019, at 16:05, annikakj notifications@github.com wrote:

In old PetDB there was a controlled vocabulary list which E-Checker read against, thus combinations that were not in this list were flagged as potential errors. DM need this functionality in new Admin/E-checker. I need to have items checked against a controlled list (E-Checker) and capability to add new combinations that will be allowed as well as a place to look up existing combinations (Admin).

Examples of the list:

ITEM_CODE METHOD_CODE AB XRF AB CALC AB PC AC CALC ACC CALC ACC PC ACT PC ALM CALC ALT PC AMG CALC AMPH XRF AMPH CALC AMPH Unknown AMPH PC AN CALC ANAL CALC AND CALC ANDR CALC ANH PC APA CALC APA PC AUG PC Ag ICPMS Ag ICPAES Ag FPHOT Ag RNAA Ag LA-ICPMS Ag Unknown — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub, or mute the thread.

[kelseymarkey]

@klehnert55 I agree, but based on my experience with Evelin and Sruti it is necessary to have controlled lists when teaching people (early-scientists/students) how to do our data entry. Considering the number of students that we expect to involve in AstroMaterials, I wonder if there is a way to keep these lists but not put so much time into the maintenance of them.

[annikakj]

I am concerned about the loss of checking mechanisms. For example, it can be weeks between data sets that give various kinds of precision data. When these papers come in I frequently refer back to the controlled list to remind myself of the proper format Without this we could get: 2S-REL, 2-S-REL, 2SREL, 2s-rel, 2srel etc etc etc. The same goes for units, especially ones like ccstp/g. Having control lists is one of the hallmarks of our high data quality and carefully curated data.

Annika

On Jan 22, 2019, at 11:51 AM, Kelsey Markey notifications@github.com wrote:

@klehnert55 https://github.com/klehnert55 I agree, but based on my experience with Evelin and Sruti it is necessary to have controlled lists when teaching people (early-scientists/students) how to do our data entry. Considering the number of students that we expect to involve in AstroMaterials, I wonder if there is a way to keep these lists but not put so much time into the maintenance of them.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/iedadata/EarthChemAdmin2/issues/9#issuecomment-456473818, or mute the thread https://github.com/notifications/unsubscribe-auth/ApwsVlFjMWjgkQsDA_VFHG4NK4QAUsmUks5vF0GMgaJpZM4aM_ka.

[sparkji]

From technique view, a new table should be added to maintain the relationship between method and variable, and the the relationships has been created will be used other echecker's functions. So the relationships need to be maintained long term to make the relationships are correct. There should have user role, which will be responsible to add, delete and update the relationships. So we should have clear workflows(suggestion, review, approve, add, delete or update) for that (same issues for other vocabularies).

[klehnert55]

I think that we need to check their entry carefully anyway, and maybe leave all of method related entries to the more senior DMs

Sent from my iPhone

On Jan 22, 2019, at 16:51, Kelsey Markey notifications@github.com wrote:

@klehnert55 I agree, but based on my experience with Evelin and Sruti it is necessary to have controlled lists when teaching people (early-scientists/students) how to do our data entry. Considering the number of students that we expect to involve in AstroMaterials, I wonder if there is a way to keep these lists but not put so much time into the maintenance of them.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.

[klehnert55]

I thought that the previous mail related to the variable-method relationship. I did not say that we should abandon all controlled vocabularies

Sent from my iPhone

On Jan 22, 2019, at 17:29, annikakj notifications@github.com wrote:

I am concerned about the loss of checking mechanisms. For example, it can be weeks between data sets that give various kinds of precision data. When these papers come in I frequently refer back to the controlled list to remind myself of the proper format Without this we could get: 2S-REL, 2-S-REL, 2SREL, 2s-rel, 2srel etc etc etc. The same goes for units, especially ones like ccstp/g. Having control lists is one of the hallmarks of our high data quality and carefully curated data.

Annika

On Jan 22, 2019, at 11:51 AM, Kelsey Markey notifications@github.com wrote:

@klehnert55 https://github.com/klehnert55 I agree, but based on my experience with Evelin and Sruti it is necessary to have controlled lists when teaching people (early-scientists/students) how to do our data entry. Considering the number of students that we expect to involve in AstroMaterials, I wonder if there is a way to keep these lists but not put so much time into the maintenance of them.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/iedadata/EarthChemAdmin2/issues/9#issuecomment-456473818, or mute the thread https://github.com/notifications/unsubscribe-auth/ApwsVlFjMWjgkQsDA_VFHG4NK4QAUsmUks5vF0GMgaJpZM4aM_ka.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.

[annikakj]

Attached is the last control list download we were able to download from PetDB in late 2017. From this you can see and compare what DM’s had access to in terms of lists where we could look things up vs what is currently on Admin. PetDB_CL_Y7Q1.xlsx

Annika

[kelseymarkey]

@klehnert55 I agree, method related entries might best be left to the project scientist for Astromaterials. I didn't realize you meant just the variable-method list, you're right that will be difficult to maintain for Astromaterials. Can't speak for PetDB.

[annikakj]

To be clear from my side: I am only referring to PetDB. Annika

On Jan 22, 2019, at 2:43 PM, Kelsey Markey notifications@github.com wrote:

@klehnert55 https://github.com/klehnert55 I agree, method related entries might best be left to the project scientist for Astromaterials. I didn't realize you meant just the variable-method list, you're right that will be difficult to maintain for Astromaterials. Can't speak for PetDB.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/iedadata/EarthChemAdmin2/issues/9#issuecomment-456536046, or mute the thread https://github.com/notifications/unsubscribe-auth/ApwsVvBLJdj8MA75BYyoYBABDZipgZMQks5vF2nPgaJpZM4aM_ka.

[klehnert55]

PetDB and AstroMat will use the same CVs at some point in the future, especially those that pertain to the analytical data (not the samples). We are starting an international effort to develop geochemical data conventions/standards, and CVs will naturally be part of that.

On 1/22/19 14:44, annikakj wrote:

To be clear from my side: I am only referring to PetDB. Annika

On Jan 22, 2019, at 2:43 PM, Kelsey Markey notifications@github.com wrote:

@klehnert55 https://github.com/klehnert55 I agree, method related entries might best be left to the project scientist for Astromaterials. I didn't realize you meant just the variable-method list, you're right that will be difficult to maintain for Astromaterials. Can't speak for PetDB.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/iedadata/EarthChemAdmin2/issues/9#issuecomment-456536046, or mute the thread https://github.com/notifications/unsubscribe-auth/ApwsVvBLJdj8MA75BYyoYBABDZipgZMQks5vF2nPgaJpZM4aM_ka.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/iedadata/EarthChemAdmin2/issues/9#issuecomment-456536426, or mute the thread https://github.com/notifications/unsubscribe-auth/AG9LCujDaS7H52QgcjRdfaJchOtzr9TDks5vF2oZgaJpZM4aM_ka.

-- Dr. Kerstin A. Lehnert Doherty Senior Research Scientist Director, Geoinformatics Research Group Director, Interdisciplinary Earth Data Alliance Lamont-Doherty Earth Observatory of Columbia University 61 Route 9W, Palisades, NY, 10964, USA +1 (845) 365-8506

[klehnert55]

Hi Peng,

The issue is the maintenance of the list. If we have DMs who do not know much about geochemical methods, what will be the process to maintain it. As the range of analytical methods grows, the maintenance of this list will require serious curation. There is quite a bit of knowledge of analytical methods required to keep the list correct and useful.

We need to discuss this before any effort is afforded.

Kerstin

On 1/22/19 12:48, Peng wrote:

From technique view, a new table should be added to maintain the relationship between method and variable, and the the relationships has been created will be used other echecker's functions. So the relationships need to be maintained long term to make the relationships are correct. There should have user role, which will be responsible to add, delete and update the relationships. So we should have clear workflows(suggestion, review, approve, add, delete or update) for that (same issues for other vocabularies).

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/iedadata/EarthChemAdmin2/issues/9#issuecomment-456494917, or mute the thread https://github.com/notifications/unsubscribe-auth/AG9LCqJicPE8xKsq1YPUgwwEvBqbneNUks5vF07lgaJpZM4aM_ka.

-- Dr. Kerstin A. Lehnert Doherty Senior Research Scientist Director, Geoinformatics Research Group Director, Interdisciplinary Earth Data Alliance Lamont-Doherty Earth Observatory of Columbia University 61 Route 9W, Palisades, NY, 10964, USA +1 (845) 365-8506