Closed further-reading closed 2 months ago
In the current version of scrapy, the code below breaks due to spider start time being timezone aware https://github.com/scrapinghub/spidermon/blob/master/spidermon/contrib/scrapy/monitors/monitors.py#L497
now = datetime.datetime.utcnow() start_time = self.data.stats.get("start_time") duration = now - start_time
It currently throws the following error:
TypeError: can't subtract offset-native and offset-aware datetimes
Backwards compatible Fix: add try/except to add utc timezone to now timestamp when error thrown.
In the current version of scrapy, the code below breaks due to spider start time being timezone aware https://github.com/scrapinghub/spidermon/blob/master/spidermon/contrib/scrapy/monitors/monitors.py#L497
It currently throws the following error:
TypeError: can't subtract offset-native and offset-aware datetimes
Backwards compatible Fix: add try/except to add utc timezone to now timestamp when error thrown.