karan / TPB

:cloud: Python API for ThePirateBay.
MIT License
331 stars 66 forks source link

Catch urlopen exceptions when retrieving list pages #67

Open isameer opened 10 years ago

isameer commented 10 years ago

This is to prevent a non-existent URL from being fetched when iterating through results in multipage mode. For example, something like

for torrent in tpb.recent().multipage():
    do something with torrent

will throw an exception since the recent torrent listing has a maximum of 100 pages.

umazalakain commented 10 years ago

Thanks for bringing up this issue. The main problem with the PR though is that it's in the wrong place. Tha try ... except code should live in the items() function of the Paginated class.

There's also the issue of the try block being too large. A try block should always try to be the most specific possible and the exception should check for specific exceptions too. I think something along the lines of the following code would work:

# In Paginated.items, after the while True
try:
    items = super(Paginated, self).items()
except urllib2.HTTPError as error:
    if error.code == 404:
        raise StopIteration()
# the rest of the code