search

BLE-LTER / MetaEgress

R package to create Ecological Metadata Language documents from an instance of LTER-core-metabase database schema
https://BLE-LTER.github.io/MetaEgress/
6 stars 3 forks source link

implement congruency checks between data & metadata #50

Open atn38 opened 5 years ago

atn38 commented 5 years ago

the ones not yet in ECC

gastil commented 5 years ago

I was hoping the pasta checker would get around to those checks. Currently the header-match is an info-level, human-viewed check. And the enum check would have saved me grief. Are you concerned you might be duplicating effort, doing this both in MetaEgress and (in future) pasta? Tho they say errors are best caught closest to the source. And who knows when check coding will resume in pasta. So I vote yes please.

atn38 commented 5 years ago

I'm not concerned about duplicating efforts. Versions of this idea have been written again and again and have their own uses in context. MetaEgress can use its own. E.g. I wrote something for datapie, the ADC folks wrote something here: https://github.com/NCEAS/datamgmt/blob/master/R/qa_package.R

And yes I just came to grief about the enum and missing codes one motivating this issue

On Mon, Nov 4, 2019 at 4:16 PM Gastil notifications@github.com wrote:

I was hoping the pasta checker would get around to those checks. Currently the header-match is an info-level, human-viewed check. And the enum check would have saved me grief. Are you concerned you might be duplicating effort, doing this both in MetaEgress and (in future) pasta? Tho they say errors are best caught closest to the source. And who knows when check coding will resume in pasta. So I vote yes please.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/BLE-LTER/MetaEgress/issues/50?email_source=notifications&email_token=AKAZD5T76WCDV7P7AKGVIETQSCNKJA5CNFSM4JIXIAAKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEDA47GA#issuecomment-549572504, or unsubscribe https://github.com/notifications/unsubscribe-auth/AKAZD5WQX2A23L3YJ7O3TLDQSCNKJANCNFSM4JIXIAAA .

atn38 commented 5 years ago

a first attempt at the function at https://github.com/BLE-LTER/MetaEgress/blob/congruence/R/check_attributes_congruence.R

next: integrating this into create_entity and issue warnings/errors as appropriate. plus issue more detailed messages, think about a way to check for presence of missing codes in data not listed in metadata

atn38 commented 4 years ago

not closing this yet, since the function needs improvements