search

Benny- / Yahoo-ticker-symbol-downloader

A web scraper for ticker symbols from yahoo finance
https://pypi.python.org/pypi/Yahoo-ticker-downloader/
Other
247 stars 87 forks source link

Program cannot download data #62

Open RafaBarneto opened 3 years ago

RafaBarneto commented 3 years ago

Since last week, the program is unable to access the yahoo finance data; retry 5 times the request and finally get a 404 Client error. Is there any change in yahoo access?, how can I solve this?. Anyone knows alternative ways for getting tickers symbols for free? Attach the terminal output. error.docx

RobinTTY commented 3 years ago

It appears to me they may have banned the user_agent defined in SimpleSymbolDownloader.py:

https://github.com/Benny-/Yahoo-ticker-symbol-downloader/blob/305835730773ae4000f38dc047bbe4cc41f782f4/ytd/SimpleSymbolDownloader.py#L9

If you change the user agent to something else the script will work. I noticed though that it will also fail again after some time, so the API may be limiting excessive requests.

RafaBarneto commented 3 years ago

Thank you Robin,

I tried your suggestion, but it is still failing. The problem I face is that the program try the 5 times to restart the download, each one with 5 times the previous duration, and finally it stop without any download. Best regards.

Rafa BC

El 14 ago 2021, a las 6:02, Robin Müller @.***> escribió:

It appears to me they may have banned the user_agent defined in SimpleSymbolDownloader.py:

https://github.com/Benny-/Yahoo-ticker-symbol-downloader/blob/305835730773ae4000f38dc047bbe4cc41f782f4/ytd/SimpleSymbolDownloader.py#L9-L11 https://github.com/Benny-/Yahoo-ticker-symbol-downloader/blob/305835730773ae4000f38dc047bbe4cc41f782f4/ytd/SimpleSymbolDownloader.py#L9-L11 If you change the user agent to something else the script will work.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/Benny-/Yahoo-ticker-symbol-downloader/issues/62#issuecomment-898813077, or unsubscribe https://github.com/notifications/unsubscribe-auth/ASW4UPELTK3HCR2LJMFARXDT4XTDTANCNFSM472GEV6Q. Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&utm_campaign=notification-email.

RobinTTY commented 3 years ago

If you haven't tried yet you can take a look at https://github.com/shilewenuw/get_all_tickers. Maybe that works.

RafaBarneto commented 3 years ago

I know it, and use it for USA main markets; the problem with that is the limited scope of markets it cover. At least I am not be able to get Europe tickers using it. Yahoo-ticker-downloader is, in theory, the best way (I know) to recover the bigger number of ticker worldwide. Thanks and regards.

Rafa BC

El 14 ago 2021, a las 16:56, Robin Müller @.***> escribió:

If you haven't tried yet you can take a look at https://github.com/shilewenuw/get_all_tickers https://github.com/shilewenuw/get_all_tickers. Maybe that works.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/Benny-/Yahoo-ticker-symbol-downloader/issues/62#issuecomment-898904408, or unsubscribe https://github.com/notifications/unsubscribe-auth/ASW4UPBWKKFSG3YX6WZCTILT4Z7YZANCNFSM472GEV6Q. Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&utm_campaign=notification-email.

RafaBarneto commented 3 years ago

Hi Robin,

I am trying to use get_all_tickers as you suggested, but always receive the following error:

ParserError: Error tokenizing data. C error: Expected 1 fields in line 5, saw 47

Do you know any solution?. Thanks and regards.

Rafa BC

El 14 ago 2021, a las 16:56, Robin Müller @.***> escribió:

If you haven't tried yet you can take a look at https://github.com/shilewenuw/get_all_tickers https://github.com/shilewenuw/get_all_tickers. Maybe that works.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/Benny-/Yahoo-ticker-symbol-downloader/issues/62#issuecomment-898904408, or unsubscribe https://github.com/notifications/unsubscribe-auth/ASW4UPBWKKFSG3YX6WZCTILT4Z7YZANCNFSM472GEV6Q. Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&utm_campaign=notification-email.

RobinTTY commented 3 years ago

You can find a solution here as far as I can see, I didn't try it myself though: https://github.com/shilewenuw/get_all_tickers/issues/15

RafaBarneto commented 3 years ago

Thank you Robin,

It is ok, it works, but as I supposed, it only download tickers from USA exchanges. The filters for region, country, etc means the origin of the company. What I am looking for is a way to download tickers for Europe countries exchanges. Many thanks, anyhow. Regards.

Rafa BC

El 14 ago 2021, a las 16:56, Robin Müller @.***> escribió:

If you haven't tried yet you can take a look at https://github.com/shilewenuw/get_all_tickers https://github.com/shilewenuw/get_all_tickers. Maybe that works.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/Benny-/Yahoo-ticker-symbol-downloader/issues/62#issuecomment-898904408, or unsubscribe https://github.com/notifications/unsubscribe-auth/ASW4UPBWKKFSG3YX6WZCTILT4Z7YZANCNFSM472GEV6Q. Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&utm_campaign=notification-email.

kyi87 commented 2 years ago

I got the same Problem when i use the script:

Invocation:

py YahooTickerDownloader.py

Error:

404 Client Error: Not Found for url: https://finance.yahoo.com/_finance_doubledown/api/resource/searchassist;searchTerm=b?device=console&returnMeta=true

Is there are solution yet?

Thanks.

RafaBarneto commented 2 years ago

Thank you Robin, It works now. Regards.

Rafa BC

El 14 ago 2021, a las 6:02, Robin Müller @.***> escribió:

It appears to me they may have banned the user_agent defined in SimpleSymbolDownloader.py:

https://github.com/Benny-/Yahoo-ticker-symbol-downloader/blob/305835730773ae4000f38dc047bbe4cc41f782f4/ytd/SimpleSymbolDownloader.py#L9-L11 https://github.com/Benny-/Yahoo-ticker-symbol-downloader/blob/305835730773ae4000f38dc047bbe4cc41f782f4/ytd/SimpleSymbolDownloader.py#L9-L11 If you change the user agent to something else the script will work.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/Benny-/Yahoo-ticker-symbol-downloader/issues/62#issuecomment-898813077, or unsubscribe https://github.com/notifications/unsubscribe-auth/ASW4UPELTK3HCR2LJMFARXDT4XTDTANCNFSM472GEV6Q. Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&utm_campaign=notification-email.