search

atreadw1492 / yahoo_fin

Scrape stock price history from new (Spring 2017) Yahoo Finance layout
MIT License
292 stars 125 forks source link

Getting '403 forbidden' responses #61

Closed erickmartinez closed 3 years ago

erickmartinez commented 3 years ago

Hi,

All of the sudden, the get_data function became useless due to a bad response from yahoo finance API. I constructed the URL manually and was able to get the data from the browser, but python responses still keep returning 403 unless I add the headers:

headers = {'User-Agent': 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11_5) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/50.0.2661.102 Safari/537.36'}
result = requests.get(site, params=params, headers=headers)

I don't know if you are planning to release a new version soon, and whether yahoo will eventually block this user agent fix, but in case it is useful, would it be possible to get a handle on the request headers within your code?

Thanks!

erickmartinez commented 3 years ago

I noticed that some other functions in stock_info.py are already passing headers to the requests (headers = {'User-agent': 'Mozilla/5.0'}). I think it is a matter of applying this change to the rest of the functions to get this package working.

khorton commented 3 years ago

headers=headers

Hi,

All of the sudden, the get_data function became useless due to a bad response from yahoo finance API. I constructed the URL manually and was able to get the data from the browser, but python responses still keep returning 403 unless I add the headers:

headers = {'User-Agent': 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11_5) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/50.0.2661.102 Safari/537.36'}
result = requests.get(site, params=params, headers=headers)

I don't know if you are planning to release a new version soon, and whether yahoo will eventually block this user agent fix, but in case it is useful, would it be possible to get a handle on the request headers within your code?

Thanks!

This change fixed the problem for me. Thanks!

atreadw1492 commented 3 years ago

@khorton @erickmartinez I just pushed a patch for this to PyPI. If you download this newest version - 0.8.9.1, it should resolve these issues.