search

humlab-sead / sead_change_control

Sane SEAD change control using Sqitch.
1 stars 0 forks source link

Import Bugs database v20200219 #61

Closed roger-mahler closed 4 years ago

roger-mahler commented 4 years ago

New data import of Bugs data. This will be the first incremental update of the Bugs data.

roger-mahler commented 4 years ago

Created target database sead_staging_bugs:

./copydb.sh --source sead_production --target sead_staging_bugs --force --sync-sequences
roger-mahler commented 4 years ago

Started import:

$ ./bugsimport --host seadserv.humlab.umu.se --database sead_staging_bugs --user humlab_admin > run20200313.log 2>&1
roger-mahler commented 4 years ago

Execution time: 66 minutes.

roger-mahler commented 4 years ago 
select bugs_table, message, count(*)
from bugs_import.bugs_errors
where change_date::date = '2020-03-13'
group by bugs_table, message
Bugs Table Message Count
TAttributes No attribute value 2
TBiology No data provided 9
TCountsheet Exception: could not execute statement; ... 27 1
TDatesCalendar No dating method found 21
TDatesCalendar Too many uncertainties of same type for a single sample. 20
TDatesPeriod No period found for code 192
TDatesPeriod No sample found for code 2
TDistrib Cannot add empty distribution text 2
TEcoDefKoch No group found for code 125
TFossil No sample found for code 11
TKeys Keys without content not permitted 2
TKeys No reference found for ref: null 1
TLab No country found 1
TPeriods No type set for period 24
TRDBCodes Sead data matches values 27
TSample Sead data has been updated since last bugs import 27
TSample X or Y values is probably an error 2
TSite Bugs data is updated but updates are disallowed. 12
TSpeciesAssociations Association type not found 18
TSynonym No reference found 1
TSynonym No synonym genus specified 13
TTaxoNotes No note provided 6
roger-mahler commented 4 years ago

The incremental data set of the @2020.02 bugs import of BugsCEP v20200219 has been compiled into change request 20200313_DML_BUGS_UPDATE_20200219.

This data needs a sanity check.

The number of updates:

Capture
roger-mahler commented 4 years ago

Are these numbers reasonable @visead?

visead commented 4 years ago

Yes, they look reasonable in relation to the changes I made to the sample dates and depths, as well as new data from Paul.

roger-mahler commented 1 year ago

CR 20200313_DML_BUGS_IMPORT_PUBLIC_20200219