search

CivicTechTO / ttc_subway_times

A scraper to grab and publish TTC subway arrival times.
GNU General Public License v3.0
40 stars 30 forks source link

Consolidate function seems to be failing often #65

Closed radumas closed 4 years ago

radumas commented 5 years ago

It has only been successful 3 times in the last month... when it should be running daily.

image

radumas commented 5 years ago

It's timing out image

radumas commented 5 years ago

Tried increasing memory size, which might speed it up, and doubling time-out time.

image

radumas commented 4 years ago

Some recent change to the consolidation function appears to bugging out

[ERROR] TypeError: join() argument must be str or bytes, not 'NoneType'
Traceback (most recent call last):
  File "/var/task/src/ttc_api_scraper/consolidate.py", line 123, in handler
    _consolidate()
  File "/var/task/src/ttc_api_scraper/consolidate.py", line 62, in _consolidate
    scrape_path = os.path.join(dir, consoli_date)
  File "/var/lang/lib/python3.7/posixpath.py", line 94, in join
    genericpath._check_arg_types('join', a, *p)
  File "/var/lang/lib/python3.7/genericpath.py", line 149, in _check_arg_types
    (funcname, s.__class__.__name__)) from None
radumas commented 4 years ago

Just deployed fix.