list index out of range

[danthegoodman1]

Sometimes I get the following error:

IndexError                                Traceback (most recent call last)
Input In [111], in <cell line: 53>()
     47     # personSearch = search("senate {} site:ballotpedia.org".format(fullName))
     48     # for result in personSearch:
     49     #     # get the first one
     50     #     print(result)
     51     #     return result
     53 for fullName in politicians:
---> 54     personUrl = getBallotPediaUrl(fullName)
     55     # Get the page content
     56     r = requests.get(personUrl, allow_redirects=True, headers={
     57         "User-Agent": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/104.0.0.0 Safari/537.36"
     58     })

Input In [111], in getBallotPediaUrl(fullName)
     42 def getBallotPediaUrl(fullName):
     43     print('searching', fullName)
---> 44     personSearch = ddg("{} site:ballotpedia.org".format(fullName), max_results=1)
     45     print(personSearch)
     46     return personSearch[0]['href']

File /opt/conda/lib/python3.9/site-packages/duckduckgo_search.py:39, in ddg(keywords, region, safesearch, time, max_results, **kwargs)
     36     if counter >= max_results:
     37         return results
---> 39 next_page = tree.xpath('.//div[@class="nav-link"]')[-1] 
     40 names = next_page.xpath('.//input[@type="hidden"]/@name')
     41 values = next_page.xpath('.//input[@type="hidden"]/@value')

IndexError: list index out of range

I imagine this is due to some rate limit page (that I do not see in the browser), but I have added the suggest 0.75 second sleep. Is this what is expected when you exceed rate limits?

[deedy5]

I think you are using an old version. You need to upgrade the package:

pip install -U duckduckgo_search

[danthegoodman1]

Yeah wow, not sure how this happened from a pip install duckduckgo_search:

python -c "import duckduckgo_search; print(duckduckgo_search.__version__)"
0.4

[deedy5]

-U

[danthegoodman1]

Yeah not sure why it initially pulled 0.4 though is what I’m saying. Got it working now!