open-contracting / kingfisher-collect

Downloads OCDS data and stores it on disk
https://kingfisher-collect.readthedocs.io
BSD 3-Clause "New" or "Revised" License
13 stars 12 forks source link

pakistan_ppra_api: IndexError: list index out of range #970

Closed sentry-io[bot] closed 1 year ago

sentry-io[bot] commented 1 year ago

Sentry Issue: KINGFISHER-COLLECT-2G

IndexError: list index out of range
(3 additional frame(s) were not displayed)
...
  File "scrapy/utils/python.py", line 353, in __next__
    return next(self.data)
  File "kingfisher_scrapy/spidermiddlewares.py", line 267, in process_spider_exception
    raise exception
  File "scrapy/core/spidermw.py", line 56, in _evaluate_iterable
    for r in iterable:
  File "kingfisher_scrapy/util.py", line 104, in wrapper
    yield from decorated(self, response, **kwargs)
  File "kingfisher_scrapy/spiders/pakistan_ppra_api.py", line 34, in parse_list
    urls = json.loads(response.xpath('//body//text()').getall()[6].replace(",\r\n\r\nhttps://www.ppra.org.pk", ""))

Spider error processing %(request)s (referer: %(referer)s)
yolile commented 1 year ago

I'm not sure what the problem was, I started a new crawl, and it is working now. cc @odscrachel