GSS_data/Migration/Scotland-overseas #99 failed

[ajtucker]

Build 'GSS_data/Migration/Scotland-overseas' is failing!

Last 50 lines of build output:

[...truncated 510 lines...]
350. duplicate_key. Row: 18018. S92000003,nrs/overseas,2016-06-30T00:00:00/P1Y,M,all,outflow,Count,People
351. duplicate_key. Row: 18110. S92000003,nrs/overseas,2017-06-30T00:00:00/P1Y,M,all,outflow,Count,People
352. duplicate_key. Row: 18202. S92000003,nrs/overseas,2001-06-30T00:00:00/P1Y,M,all,balance,Count,People
353. duplicate_key. Row: 18294. S92000003,nrs/overseas,2002-06-30T00:00:00/P1Y,M,all,balance,Count,People
354. duplicate_key. Row: 18386. S92000003,nrs/overseas,2003-06-30T00:00:00/P1Y,M,all,balance,Count,People
355. duplicate_key. Row: 18478. S92000003,nrs/overseas,2004-06-30T00:00:00/P1Y,M,all,balance,Count,People
356. duplicate_key. Row: 18570. S92000003,nrs/overseas,2005-06-30T00:00:00/P1Y,M,all,balance,Count,People
357. duplicate_key. Row: 18662. S92000003,nrs/overseas,2006-06-30T00:00:00/P1Y,M,all,balance,Count,People
358. duplicate_key. Row: 18754. S92000003,nrs/overseas,2007-06-30T00:00:00/P1Y,M,all,balance,Count,People
359. duplicate_key. Row: 18846. S92000003,nrs/overseas,2008-06-30T00:00:00/P1Y,M,all,balance,Count,People
360. duplicate_key. Row: 18938. S92000003,nrs/overseas,2009-06-30T00:00:00/P1Y,M,all,balance,Count,People
361. duplicate_key. Row: 19030. S92000003,nrs/overseas,2010-06-30T00:00:00/P1Y,M,all,balance,Count,People
362. duplicate_key. Row: 19122. S92000003,nrs/overseas,2011-06-30T00:00:00/P1Y,M,all,balance,Count,People
363. duplicate_key. Row: 19214. S92000003,nrs/overseas,2012-06-30T00:00:00/P1Y,M,all,balance,Count,People
364. duplicate_key. Row: 19306. S92000003,nrs/overseas,2013-06-30T00:00:00/P1Y,M,all,balance,Count,People
365. duplicate_key. Row: 19398. S92000003,nrs/overseas,2014-06-30T00:00:00/P1Y,M,all,balance,Count,People
366. duplicate_key. Row: 19490. S92000003,nrs/overseas,2015-06-30T00:00:00/P1Y,M,all,balance,Count,People
367. duplicate_key. Row: 19582. S92000003,nrs/overseas,2016-06-30T00:00:00/P1Y,M,all,balance,Count,People
368. duplicate_key. Row: 19674. S92000003,nrs/overseas,2017-06-30T00:00:00/P1Y,M,all,balance,Count,People
[Pipeline] }
[Pipeline] // ansiColor
[Pipeline] }
[Pipeline] // script
[Pipeline] }
$ docker stop --time=1 29c0c5cfb8e7906e2d1839bfa8026221c7792bc63acdd57044c530ce2e1c25ad
$ docker rm -f 29c0c5cfb8e7906e2d1839bfa8026221c7792bc63acdd57044c530ce2e1c25ad
[Pipeline] // withDockerContainer
[Pipeline] }
[Pipeline] // stage
[Pipeline] stage
[Pipeline] { (Upload Tidy Data)
Stage "Upload Tidy Data" skipped due to earlier failure(s)
[Pipeline] }
[Pipeline] // stage
[Pipeline] stage
[Pipeline] { (Test draft dataset)
Stage "Test draft dataset" skipped due to earlier failure(s)
[Pipeline] }
[Pipeline] // stage
[Pipeline] stage
[Pipeline] { (Publish)
Stage "Publish" skipped due to earlier failure(s)
[Pipeline] }
[Pipeline] // stage
[Pipeline] stage
[Pipeline] { (Declarative: Post Actions)
[Pipeline] script
[Pipeline] {
[Pipeline] step

Changes since last successful build:

• [Alex Tucker] d304a5acc8766e52976306021fa62c6ab7bae04c - Add template transformation pipeline.

• [Alex Tucker] 7eda5c2f5f1a251462cd0a2820b201263fd36e11 - Add getDraftsetEndpoint

• [Alex Tucker] 7c9c4e98b4ee0c634379ab5253c3293b426e7301 - Add SPARQL tests step and publish results.

• [Alex Tucker] 806951bf2ee882eb722ed68d14041b7f0e928fea - SPARQL tests need union-with-live=true

• [Alex Tucker] d747d17b909dea7f689436a623263ff531631c94 - Don't trigger GDP-tests as they're now run on the unpublished draftset.

• [Alex Tucker] 57b7cae64f2422e744b548d8533c87ea96b0c7b0 - Template for multiple datasets.

• [Alex Tucker] 554340c2f38e69909c77daea277188852640b339 - Disambiguate use of 'dataset'

• [Alex Tucker] 1452178f3d07debcb12de21cb3782810370ff2c0 - Use $refFamily

• [Alex Tucker] 0ab4797047cdc3a26e01fe2cfd6da3da461c57f3 - Try to always update Trello card.

• [Alex Tucker] 2c3a2d2dcad68f95080061335209f3d7241b6af9 - If main.py exists, convert to notebook using jupytext.

• [Alex Tucker] 1acd29462408540f9fbc087e739c4dd458b20af7 - Convert any .py files to .ipynb using jupytext.

No changes No changes No changes No changes No changes No changes No changes No changes No changes No changes No changes No changes No changes

• [vtula2000] 2534cb329ceffb69653ada0a472812ca2ccb10e2 - Data Baker errors

• [vtula2000] a07e390200b402772a26040edf55b3a328f3c17f - Error

• [vtula2000] 51b6e97a17334f84f7452df5a77e3912aa03e051 - Errors

• [Alex Tucker] 0ab9caa46fd9a9ce37f92f6ff1886ef7d0fad336 - Use GitHub issues rather than updating a Trello card.

• [Alex Tucker] b57d083813b089ab8107b2342f26cf94bf589f8b - Use gsscogs Docker images.

No changes

• [Alex Tucker] cc2bca87d857753755272fa5aaa08034ec2428f6 - Add template for ref_* pipelines.

• [Alex Tucker] fd491cc35f5f1a690f227435207d77378bce174e - Also validate columns.csv and components.csv.

No changes

• [Alex Tucker] 5f5d4b46d7d338090ecfa2d02ede7ae8c0a498da - Ignore RequestAborted exceptions in empty_cache/sync_search calls.

• [Alex Tucker] ef905fd5f2db85565dc4eb0bde049534e4846b0f - Use --no-verbose to cut down line length of csvlint output, so it can be

• [mikelovesbooks] e5c21ac5739e95253ebfc199f10a5e6b87f2e024 - update to new website taxonomy, update explicit row numbers

No changes No changes No changes No changes

• [vtula2000] 8650cfd5df2eb4c43840368b4534201206f95d84 - .py and schema update

View full output

[ajtucker]

Build 'GSS_data/Migration/Scotland-overseas' is failing!

Last 50 lines of build output:

[...truncated 156 lines...]
------------------

---------------------------------------------------------------------------
IndexError                                Traceback (most recent call last)
~var/jenkins_home/workspace/GSS_data/Migration/Scotland-overseas/migration-by-age-2001-to-2017-females.ipynb in <module>
----> 1 tab = [t for t in tabs if t.name == 'SYOA Feales (2001-)'][0]
      2 cell = tab.filter('Year')
      3 age = cell.fill(RIGHT).is_not_blank().is_not_whitespace() | cell.shift(0,1).fill(RIGHT).is_not_blank().is_not_whitespace()
      4 year = cell.shift(0,1).expand(DOWN).is_not_blank().is_not_whitespace()
      5 flow = tab.filter(contains_string('migration')).is_not_blank().is_not_whitespace()

IndexError: list index out of range
IndexError: list index out of range

[Pipeline] }
[Pipeline] // ansiColor
[Pipeline] }
[Pipeline] // script
[Pipeline] }
$ docker stop --time=1 af447428204334f1d6787fa76df7fde30e40604ce0114fddb4469ea3b9a8433d
$ docker rm -f af447428204334f1d6787fa76df7fde30e40604ce0114fddb4469ea3b9a8433d
[Pipeline] // withDockerContainer
[Pipeline] }
[Pipeline] // stage
[Pipeline] stage
[Pipeline] { (Validate CSV)
Stage "Validate CSV" skipped due to earlier failure(s)
[Pipeline] }
[Pipeline] // stage
[Pipeline] stage
[Pipeline] { (Upload Tidy Data)
Stage "Upload Tidy Data" skipped due to earlier failure(s)
[Pipeline] }
[Pipeline] // stage
[Pipeline] stage
[Pipeline] { (Test draft dataset)
Stage "Test draft dataset" skipped due to earlier failure(s)
[Pipeline] }
[Pipeline] // stage
[Pipeline] stage
[Pipeline] { (Publish)
Stage "Publish" skipped due to earlier failure(s)
[Pipeline] }
[Pipeline] // stage
[Pipeline] stage
[Pipeline] { (Declarative: Post Actions)
[Pipeline] script
[Pipeline] {
[Pipeline] step

Changes since last successful build:

• [Alex Tucker] d304a5acc8766e52976306021fa62c6ab7bae04c - Add template transformation pipeline.

• [Alex Tucker] 7eda5c2f5f1a251462cd0a2820b201263fd36e11 - Add getDraftsetEndpoint

• [Alex Tucker] 7c9c4e98b4ee0c634379ab5253c3293b426e7301 - Add SPARQL tests step and publish results.

• [Alex Tucker] 806951bf2ee882eb722ed68d14041b7f0e928fea - SPARQL tests need union-with-live=true

• [Alex Tucker] d747d17b909dea7f689436a623263ff531631c94 - Don't trigger GDP-tests as they're now run on the unpublished draftset.

• [Alex Tucker] 57b7cae64f2422e744b548d8533c87ea96b0c7b0 - Template for multiple datasets.

• [Alex Tucker] 554340c2f38e69909c77daea277188852640b339 - Disambiguate use of 'dataset'

• [Alex Tucker] 1452178f3d07debcb12de21cb3782810370ff2c0 - Use $refFamily

• [Alex Tucker] 0ab4797047cdc3a26e01fe2cfd6da3da461c57f3 - Try to always update Trello card.

• [Alex Tucker] 2c3a2d2dcad68f95080061335209f3d7241b6af9 - If main.py exists, convert to notebook using jupytext.

• [Alex Tucker] 1acd29462408540f9fbc087e739c4dd458b20af7 - Convert any .py files to .ipynb using jupytext.

No changes No changes No changes No changes No changes No changes No changes No changes No changes No changes No changes No changes No changes

• [vtula2000] 2534cb329ceffb69653ada0a472812ca2ccb10e2 - Data Baker errors

• [vtula2000] a07e390200b402772a26040edf55b3a328f3c17f - Error

• [vtula2000] 51b6e97a17334f84f7452df5a77e3912aa03e051 - Errors

• [Alex Tucker] 0ab9caa46fd9a9ce37f92f6ff1886ef7d0fad336 - Use GitHub issues rather than updating a Trello card.

• [Alex Tucker] b57d083813b089ab8107b2342f26cf94bf589f8b - Use gsscogs Docker images.

No changes

• [Alex Tucker] cc2bca87d857753755272fa5aaa08034ec2428f6 - Add template for ref_* pipelines.

• [Alex Tucker] fd491cc35f5f1a690f227435207d77378bce174e - Also validate columns.csv and components.csv.

No changes

• [Alex Tucker] 5f5d4b46d7d338090ecfa2d02ede7ae8c0a498da - Ignore RequestAborted exceptions in empty_cache/sync_search calls.

• [Alex Tucker] ef905fd5f2db85565dc4eb0bde049534e4846b0f - Use --no-verbose to cut down line length of csvlint output, so it can be

• [mikelovesbooks] e5c21ac5739e95253ebfc199f10a5e6b87f2e024 - update to new website taxonomy, update explicit row numbers

No changes No changes No changes No changes

• [vtula2000] 8650cfd5df2eb4c43840368b4534201206f95d84 - .py and schema update

• [mikelovesbooks] 60a9955031bd2846e02ccde85223f07ac1eca41d - moved to databaker to deal with duplication from explicit row slicing

• [mikelovesbooks] 05642c0ba31280fe2f68a024000c25b1cdae8f60 - add source column, add temporary fix for precision differnces

View full output

[ajtucker]

Build 'GSS_data/Migration/Scotland-overseas' is failing!

Last 50 lines of build output:

[...truncated 231 lines...]
------------------

---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-27-de2f78ef2b19> in <module>
     42         else:
     43             raise ValueError("Aborting. We have duplicate numbers that appear to be differentiated by "
---> 44                         "more than just rounding off, these: {}".format(",".join([str(x) for x in both_values])))
     45 
     46     # Now we compare precision and drop the least precise

ValueError: Aborting. We have duplicate numbers that appear to be differentiated by more than just rounding off, these: 12000,11985
ValueError: Aborting. We have duplicate numbers that appear to be differentiated by more than just rounding off, these: 12000,11985

[Pipeline] }
[Pipeline] // ansiColor
[Pipeline] }
[Pipeline] // script
[Pipeline] }
$ docker stop --time=1 4de491a9d5e255a6e843e4706cf48ca1268a795876ef2dd5b64757e60f180718
$ docker rm -f 4de491a9d5e255a6e843e4706cf48ca1268a795876ef2dd5b64757e60f180718
[Pipeline] // withDockerContainer
[Pipeline] }
[Pipeline] // stage
[Pipeline] stage
[Pipeline] { (Validate CSV)
Stage "Validate CSV" skipped due to earlier failure(s)
[Pipeline] }
[Pipeline] // stage
[Pipeline] stage
[Pipeline] { (Upload Tidy Data)
Stage "Upload Tidy Data" skipped due to earlier failure(s)
[Pipeline] }
[Pipeline] // stage
[Pipeline] stage
[Pipeline] { (Test draft dataset)
Stage "Test draft dataset" skipped due to earlier failure(s)
[Pipeline] }
[Pipeline] // stage
[Pipeline] stage
[Pipeline] { (Publish)
Stage "Publish" skipped due to earlier failure(s)
[Pipeline] }
[Pipeline] // stage
[Pipeline] stage
[Pipeline] { (Declarative: Post Actions)
[Pipeline] script
[Pipeline] {
[Pipeline] step

Changes since last successful build:

• [Alex Tucker] d304a5acc8766e52976306021fa62c6ab7bae04c - Add template transformation pipeline.

• [Alex Tucker] 7eda5c2f5f1a251462cd0a2820b201263fd36e11 - Add getDraftsetEndpoint

• [Alex Tucker] 7c9c4e98b4ee0c634379ab5253c3293b426e7301 - Add SPARQL tests step and publish results.

• [Alex Tucker] 806951bf2ee882eb722ed68d14041b7f0e928fea - SPARQL tests need union-with-live=true

• [Alex Tucker] d747d17b909dea7f689436a623263ff531631c94 - Don't trigger GDP-tests as they're now run on the unpublished draftset.

• [Alex Tucker] 57b7cae64f2422e744b548d8533c87ea96b0c7b0 - Template for multiple datasets.

• [Alex Tucker] 554340c2f38e69909c77daea277188852640b339 - Disambiguate use of 'dataset'

• [Alex Tucker] 1452178f3d07debcb12de21cb3782810370ff2c0 - Use $refFamily

• [Alex Tucker] 0ab4797047cdc3a26e01fe2cfd6da3da461c57f3 - Try to always update Trello card.

• [Alex Tucker] 2c3a2d2dcad68f95080061335209f3d7241b6af9 - If main.py exists, convert to notebook using jupytext.

• [Alex Tucker] 1acd29462408540f9fbc087e739c4dd458b20af7 - Convert any .py files to .ipynb using jupytext.

No changes No changes No changes No changes No changes No changes No changes No changes No changes No changes No changes No changes No changes

• [vtula2000] 2534cb329ceffb69653ada0a472812ca2ccb10e2 - Data Baker errors

• [vtula2000] a07e390200b402772a26040edf55b3a328f3c17f - Error

• [vtula2000] 51b6e97a17334f84f7452df5a77e3912aa03e051 - Errors

• [Alex Tucker] 0ab9caa46fd9a9ce37f92f6ff1886ef7d0fad336 - Use GitHub issues rather than updating a Trello card.

• [Alex Tucker] b57d083813b089ab8107b2342f26cf94bf589f8b - Use gsscogs Docker images.

No changes

• [Alex Tucker] cc2bca87d857753755272fa5aaa08034ec2428f6 - Add template for ref_* pipelines.

• [Alex Tucker] fd491cc35f5f1a690f227435207d77378bce174e - Also validate columns.csv and components.csv.

No changes

• [Alex Tucker] 5f5d4b46d7d338090ecfa2d02ede7ae8c0a498da - Ignore RequestAborted exceptions in empty_cache/sync_search calls.

• [Alex Tucker] ef905fd5f2db85565dc4eb0bde049534e4846b0f - Use --no-verbose to cut down line length of csvlint output, so it can be

• [mikelovesbooks] e5c21ac5739e95253ebfc199f10a5e6b87f2e024 - update to new website taxonomy, update explicit row numbers

No changes No changes No changes No changes

• [vtula2000] 8650cfd5df2eb4c43840368b4534201206f95d84 - .py and schema update

• [mikelovesbooks] 60a9955031bd2846e02ccde85223f07ac1eca41d - moved to databaker to deal with duplication from explicit row slicing

• [mikelovesbooks] 05642c0ba31280fe2f68a024000c25b1cdae8f60 - add source column, add temporary fix for precision differnces

• [mikelovesbooks] 1f5bf662ed200a206cee9f7fd2d29ee720decf83 - fix typo

View full output

[ajtucker]

Build was fixed!