IgnoredAmbience / yahoo-group-archiver

Scrapes and archives a Yahoo groups email archives, photo galleries and file contents using the non-public API
MIT License
93 stars 46 forks source link

Error after calendar event retries are exceeded #67

Closed ZigmundRat closed 4 years ago

ZigmundRat commented 4 years ago

2019-10-25 20:47:59.632 EDT INFO archive_calendar Trying to get events between 20010130 and 20031027 2019-10-25 20:47:59.634 EDT INFO requests.packages.urllib3.connectionpool Starting new HTTPS connection (10): calendar.yahoo.com 2019-10-25 20:48:00.141 EDT DEBUG requests.packages.urllib3.connectionpool "GET /ws/v3/users/@@group_c06cafdb-8bf9-43ce-993f-0183b21d98c7/calendars/events/?format=json&dtstart=20010130&dtend=20031027&wssid=OJ8rIFSQ6TE HTTP/1.1" 500 None 2019-10-25 20:48:00.142 EDT ERROR archive_calendar HTTP error (sleeping before retry, try 9: 500 Server Error: Server Error for url: https://calendar.yahoo.com/ws/v3/users/@@group_c06cafdb-8bf9-43ce-993f-0183b21d98c7/calendars/events/?format=json&dtstart=20010130&dtend=20031027&wssid=OJ8rIFSQ6TE Traceback (most recent call last): File "./yahoo.py", line 707, in archive_calendar(yga) File "./yahoo.py", line 400, in archive_calendar calContent = json.loads(calContentRaw) UnboundLocalError: local variable 'calContentRaw' referenced before assignment

Making the following change around line 400 allows the archiving to continue:

for i in range(TRIES):
        try:
            logger.info("Trying to get events between %s and %s", jsonStart, jsonEnd)
            calContentRaw = yga.download_file(calURL)
            break
        except requests.exceptions.HTTPError as err:
->          calContentRaw = None
            logger.error("HTTP error (sleeping before retry, try %d: %s", i, err)
            time.sleep(HOLDOFF)
->      if calContentRaw is None:
->          logger.error("ERROR: Couldn't retrieve calendar for this group")
->          return