humlab-sead / sead_change_control

Sane SEAD change control using Sqitch.
1 stars 0 forks source link

Data Quality Control #53

Open visead opened 4 years ago

visead commented 4 years ago

Create a quality control system that can be run when needed, primarily for when new data is imported or when system changes. Include: Record counts on key tables - before and after event

visead commented 4 years ago

Quality control query in bugsdata.mdb created 20200219 to count records in each table

QNumRecordsPerTable

SELECT [Name], DCount("","[" & [Name] & "]") AS RecCount FROM MSysObjects WHERE Type=1 AND [Name] Not Like "Msys" ORDER BY 1

result at 20200219_16:30:

Name RecCount
BuildVersionBugsdata 22
INDEX 11245
TAttributes 11962
TBiblio 6351
TBiology 37578
TbirmBEETLEdat 15696
TCountry 5
TCountsheet 1465
TDatesCalendar 2042
TDatesMethods 47
TDatesPeriod 7679
TDatesRadio 1368
TDatesTypes 4
TDistrib 41119
TEcoBugs 9057
TEcoDefBugs 22
TEcoDefGroups 6
TEcoDefKoch 125
TEcoKoch 17224
TFossil 193054
TFossilUncertainty 0
TINDEXReplacements 1072
TKeys 429
TLab 235
TLookupCountsheetContext 5
TLookupCountsheetTypes 4
TLookupMonths 12
TLookupUncertaintyType 6
TMCRNames 436
TMCRSummaryData 436
TPeriods 122
TRDB 2187
TRDBCodes 27
TRDBSystems 3
TSample 7940
TSeasonActiveAdult 9907
TSite 1316
TSiteOtherProxies 1129
TSiteRef 2027
TSpeciesAssociations 962
TSynonym 6872
TSynonymNotes 0
TTaxoNotes 10392
visead commented 4 years ago

Bugsdata record count 2020/03/20 09:25 http://bugscep.com/downloads/bugsdata.zip Added new value: All tables | 405246

Name RecCount
BuildVersionBugsdata 22
INDEX 11247
TAttributes 12025
TBiblio 6364
TBiology 37601
TbirmBEETLEdat 15696
TCountry 5
TCountsheet 1468
TDatesCalendar 2044
TDatesMethods 47
TDatesPeriod 7680
TDatesRadio 1368
TDatesTypes 4
TDistrib 41133
TEcoBugs 9057
TEcoDefBugs 22
TEcoDefGroups 6
TEcoDefKoch 125
TEcoKoch 17224
TFossil 194954
TFossilUncertainty 0
TINDEXReplacements 1072
TKeys 429
TLab 235
TLookupCountsheetContext 5
TLookupCountsheetTypes 4
TLookupMonths 12
TLookupUncertaintyType 6
TMCRNames 436
TMCRSummaryData 436
TPeriods 122
TRDB 2187
TRDBCodes 27
TRDBSystems 3
TSample 7979
TSeasonActiveAdult 9907
TSite 1319
TSiteOtherProxies 1132
TSiteRef 2031
TSpeciesAssociations 962
TSynonym 6877
TSynonymNotes 0
TTaxoNotes 11973
All tables 405246
visead commented 4 years ago

Bugsdata record count 2020/06/14 http://bugscep.com/downloads/bugsdata.zip Added new value: All tables | 408728 Major updates to TPeriod and dating evidence for >40 sites. Every valid period should now have a date range.

Name RecCount
BuildVersionBugsdata 24
INDEX 11276
TAttributes 12029
TBiblio 6422
TBiology 37990
TbirmBEETLEdat 15696
TCountry 5
TCountsheet 1484
TDatesCalendar 2213
TDatesMethods 50
TDatesPeriod 7731
TDatesRadio 1376
TDatesTypes 4
TDistrib 41630
TEcoBugs 9061
TEcoDefBugs 22
TEcoDefGroups 6
TEcoDefKoch 125
TEcoKoch 17236
TFossil 196899
TFossilUncertainty 0
TINDEXReplacements 1072
TKeys 429
TLab 236
TLookupCountsheetContext 6
TLookupCountsheetTypes 4
TLookupMonths 12
TLookupUncertaintyType 6
TMCRNames 436
TMCRSummaryData 436
TPeriods 139
TRDB 2187
TRDBCodes 27
TRDBSystems 3
TSample 8158
TSeasonActiveAdult 9907
TSite 1330
TSiteOtherProxies 1141
TSiteRef 2049
TSpeciesAssociations 967
TSynonym 6895
TSynonymNotes 0
TTaxoNotes 12009