search

Benny- / Yahoo-ticker-symbol-downloader

A web scraper for ticker symbols from yahoo finance
https://pypi.python.org/pypi/Yahoo-ticker-downloader/
Other
247 stars 87 forks source link

search term `null` returns http 503 #59

Open KIC opened 3 years ago

KIC commented 3 years ago

Once the algorithm reaches the search term null the server responses with a 503. As a consequence, the downloader basically suspends forever :-(

https://finance.yahoo.com/_finance_doubledown/api/resource/searchassist;searchTerm=null?device=console&returnMeta=true

maybe we need a list of invalid words (I could imagine that SQL keywords also make problems).

req https://finance.yahoo.com/_finance_doubledown/api/resource/searchassist;searchTerm=null?device=console&returnMeta=true
Retry attempt: 1 of 5. Sleep period: 5 seconds.
req https://finance.yahoo.com/_finance_doubledown/api/resource/searchassist;searchTerm=null?device=console&returnMeta=true
Retry attempt: 2 of 5. Sleep period: 25 seconds.
req https://finance.yahoo.com/_finance_doubledown/api/resource/searchassist;searchTerm=null?device=console&returnMeta=true
Retry attempt: 3 of 5. Sleep period: 125 seconds.
req https://finance.yahoo.com/_finance_doubledown/api/resource/searchassist;searchTerm=null?device=console&returnMeta=true
Retry attempt: 4 of 5. Sleep period: 625 seconds.
req https://finance.yahoo.com/_finance_doubledown/api/resource/searchassist;searchTerm=null?device=console&returnMeta=true
Retry attempt: 5 of 5. Sleep period: 3125 seconds.
^CSuspending downloader to disk as .pickle file