search

m-lab / scraper

Scrape experiment data off of MLab nodes and upload it to Google Cloud Storage
Apache License 2.0
5 stars 5 forks source link

Travis cron builds failing #318

Open nkinkade opened 5 years ago

nkinkade commented 5 years ago

Travis cron-based builds of this repo started failing for no apparently reason a month or two ago. There were no changes in the code. There are many errors similar to this one:

ERROR: test_main (run_scraper_test.EndToEndWithFakes)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/mock/mock.py", line 1305, in patched
    return func(*args, **keywargs)
  File "/home/travis/build/m-lab/scraper/run_scraper_test.py", line 235, in test_main
    '--max_uncompressed_size', '1024'])
  File "/home/travis/build/m-lab/scraper/run_scraper.py", line 205, in main
    args = parse_cmdline(argv[1:])
  File "/home/travis/build/m-lab/scraper/run_scraper.py", line 81, in parse_cmdline
    parents=[oauth2client.tools.argparser],
AttributeError: 'module' object has no attribute 'tools'
pboothe commented 5 years ago

Aha! We learned something from the most recent failure!

======================================================================
FAIL: test_main_with_no_data (run_scraper_test.EndToEndWithFakes)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/mock/mock.py", line 1305, in patched
    return func(*args, **keywargs)
  File "/home/travis/build/m-lab/scraper/run_scraper_test.py", line 291, in test_main_with_no_data
    'too much' % (value['maxrawfilemtimearchived'], time_since_epoch))
AssertionError: maxrawfilemtimearchived(1551311999) and time_since_epoch(1551398400) differ by too much
----------------------------------------------------------------------

And 1551398400 - 1551311999 == 86401, and 86401 = 24*60*60+1.

So this happens when the unit test straddles the midnight boundary! Now to find the fix...