search

guoguo12 / billboard-charts

Python API for downloading Billboard charts.
https://pypi.python.org/pypi/billboard.py
MIT License
389 stars 117 forks source link

Timeout on older charts #67

Open critmcdonald opened 4 years ago

critmcdonald commented 4 years ago

This error indicates it is more a problem from Billboard, but in a loop pulling charts I get significantly more timeouts on older charts, starting seriously around 2010. These aren't rate-limiting issues as I've mitigated those with time intervals and the error is different ... it seems like Billboard won't respond quickly enough. Sometimes it will grab two or three charts and then die. I can use the same code to pull recent charts with no problem, but once I try for an older year, it times out again.

Here is an example

Not sure your module can do anything about this ... more wondering if other folks have seen this.

guoguo12 commented 4 years ago

I've experienced this! One of the unit tests was flaky until I changed it to use a newer date (https://github.com/guoguo12/billboard-charts/commit/d1f26b933a1e3a4aa3794f1a7017e4bfe8595cf4).

Curious to see if anyone finds a workaround.

0xAether commented 4 years ago

I'm experiencing this too. Below is a command I'm running to scrape country music charts from the 1990's. It seems like Billboard is returning a 503 seemingly often. I've notice this when I browse out to Billboard chart pages in Chrome. Although it seems to happen much more often when using commands like the one below.

billboard.ChartData('country-songs', date=chartDate, fetch=True, timeout=None, max_retries=20)