search

colonelpanic8 / okcupyd

A Library that enables programmatic interaction with okcupid.com, using okcupid.com's private okcupid JSON API and html scraping when necessary.
MIT License
110 stars 18 forks source link

fetchable.py PaginationProcessor sometimes fails with IndexError #60

Open dubiousjim opened 9 years ago

dubiousjim commented 9 years ago

I'm not sure anymore how to reproduce this problem, I encountered it and patched it in my own installation a while back. But changing line 339 of util/fetchable.py from:

return self._current_page(tree) < self._page_count(tree)

to:

try:
    return self._current_page(tree) < self._page_count(tree)
except IndexError:
    return False

fixed some exceptions I was encountering.

colonelpanic8 commented 9 years ago

Do you remember what type of fetchable this was happening with? The IndexError means that the html element is not being found at all.

dubiousjim commented 9 years ago

Sorry, it was too long ago that I implemented this, I don't remember. Might have been on a list of questions.