search

humlab-sead / sead_clearinghouse_import

Temporary import system for SEAD Clearinghouse
0 stars 0 forks source link

Isotope: Import data to SEAD Clearinghouse #1

Closed roger-mahler closed 4 years ago

roger-mahler commented 5 years ago

Import the new isotope data to SEAD.

roger-mahler commented 5 years ago

Fixed problems:

[ x ] Added sheet alias tbl_a_e_prep_method [ x ] Renamed sheet tbl_analysis_e_prep_method to tbl_a_e_prep_method [ x ] Renamed entry tbl_a_e_prep_method to tbl_analysis_entity_prep_methods in data_table_index [ x ] Fixed typo tbl_analysis_entitiy_prep_methods in in data_table_index [ x ] Fixed typo tbl_dataset_contats in in data_table_index [ x ] Fixed naming (lowercase, xe6) in table tbl_standards [ x ] Fixed missing FK marker (table) in tbl_isotopes: isotope_measurment_id, isotope_measurment_id

roger-mahler commented 5 years ago 
WARNING Type clash: tbl_isotopes.measurment_value numeric<=>object
WARNING Type clash: tbl_isotope_types.alternativ_name character varying<=>float64
WARNING Type clash: tbl_relative_dates.notes text<=>float64
WARNING Type clash: tbl_relative_ages.abbreviation character varying<=>float64
WARNING Type clash: tbl_sites.national_site_identifier character varying<=>float64
WARNING Type clash: tbl_sites.site_location_accuracy character varying<=>float64
WARNING tbl_sample_groups has EXTRA DATA columns: Unnamed: 9, Unnamed: 10, Unnamed: 8
WARNING Type clash: tbl_physical_samples.date_sampled character varying<=>float64
WARNING tbl_datasets has EXTRA DATA columns: projekt_id
WARNING Type clash: tbl_dataset_masters.url text<=>float64
WARNING Type clash: tbl_dataset_submissions.date_submitted date<=>float64
WARNING Type clash: tbl_contacts.address_2 character varying<=>float64
WARNING Type clash: tbl_biblio.bugs_reference character varying<=>float64
WARNING Type clash: tbl_biblio.isbn character varying<=>float64
WARNING Type clash: tbl_biblio.year character varying<=>int64
WARNING Type clash: tbl_standards.isotope_ration character varying<=>float64
WARNING Type clash: tbl_standards.international_scale character varying<=>float64
WARNING Type clash: tbl_standards.accepted_ratio_xe6 character varying<=>float64
WARNING Type clash: tbl_standards.error_of_ratio character varying<=>float64
WARNING Type clash: tbl_standards.reference character varying<=>float64
MGE4 commented 5 years ago

Tar en titt på problemen så skickar jag dig en ny fil när jag fixat de problemen, låter det okej?

roger-mahler commented 5 years ago

Note:

roger-mahler commented 5 years ago

Errors and warnings from import of 20191017 data (sorry for the delay!)

WARNING Type clash: tbl_isotopes.measurment_value numeric<=>object
WARNING! Column tbl_isotopes.isotope_value_specifier_id: ends with "_id" but NOT marked as PK/FK
WARNING tbl_isotope_value_specifiers has EXTRA DATA columns: description, isotope_value_specifier_id, Name
CRITICAL ERROR Column tbl_isotopes.measurment_value has non-numeric values: Non detected
CRITICAL ERROR Table tbl_isotope_value_specifiers has no PK named "None"
ABORTED CRITICAL ERROR isotope_data_latest_20191017:
   Process ABORTED since submission does not conform to SPECIFICATION

These data tables must be added to SEAD as a change request i.e. we need to add the data to 20191012_DML_ISOTOPE_LOOKUP_DATA.

CRITICAL ERROR tbl_isotope_types new values not allowed for lookup table.
CRITICAL ERROR tbl_data_types new values not allowed for lookup table.
CRITICAL ERROR tbl_sample_location_types new values not allowed for lookup table.
CRITICAL ERROR tbl_locations new values not allowed for lookup table.
CRITICAL ERROR tbl_dataset_masters new values not allowed for lookup table.
CRITICAL ERROR tbl_contacts new values not allowed for lookup table.
CRITICAL ERROR tbl_biblio new values not allowed for lookup table.
roger-mahler commented 5 years ago

Table tbl_isotope_value_specifiers:

MGE4 commented 4 years ago

I have updated the files and added them to the designated folder, the problems should be fixed. Looking at your code, i could not find any problems that stood out to me.

MGE4 commented 4 years ago

Noticed a spelling error in "isotope_data_latest (FIX 6)", i fixed it and uploaded "isotope_data_latest (FIX 7)" with the corrections.

roger-mahler commented 4 years ago

Isotope data is imported. Remaining errors are logged in #18.