search

Benny- / Yahoo-ticker-symbol-downloader

A web scraper for ticker symbols from yahoo finance
https://pypi.python.org/pypi/Yahoo-ticker-downloader/
Other
247 stars 87 forks source link

Download does not return any stocks or indexs #18

Closed jch001 closed 7 years ago

jch001 commented 7 years ago

Hi Benny,

I have tried running the script from PyCharm for Stocks and Indexs. I haven't tried the others. But, nothing gets returned.
it may be an issue with Yahoo's webpage. as when I put the URLs in below to a browser. Nothing is returned. can you confirm please.

https://finance.yahoo.com/lookup/?s=p&r=&m=ALL&t=S&b=0 https://finance.yahoo.com/lookup/?bypass=true&s=n&r=&m=ALL&t=S&b=0

Benny- commented 7 years ago

You are right. Yahoo changed something. I will have to look into it.

I'm not yet sure when I have time to look into it again.

AlManja commented 7 years ago

Hi Benny, thank you for doing this. I just ran your scrip yesterday not knowing there is a problem. (Python 3.5.2). I did get 3 stocks though :-)

Benny- commented 7 years ago

Hi @AlManja, I am interested in how you got those 3 stocks. How long have you ran the program? I tried to run in briefly myself but got nothing.

bfendler commented 7 years ago

Not sure if this is related, but I think I see the issue. I just started looking at downloading ticker symbols and am not proficient in html or python, but it seems that "ALL" no longer works in the regional choice as you have initialized in your SymbolDownloader class, eg: h ttps://finance.yahoo.com/lookup/stocks?t=S&m=ALL&r=&s=a&b=20. Secure or insecure will not work (http(s)). type='stocks' doesn't seem to be right either.

So, main point, if you replace 'stocks' with 's' and 'ALL' with the country codes, it works. You might just need to create another parser. Unfortunately:

If you do that with the US country codes, you get a blank page. There's no error, but the page looks like it's just not giving things up. Looks like all US associated provinces as well (VI), (UM), see here: http://www.nationsonline.org/oneworld/country_code_list.htm, have the sam issue. Other countries seem to work including Britain (GB).

Benny- commented 7 years ago

Cool, so https://finance.yahoo.com/lookup/stocks?t=S&m=GB&r=&s=a&b=20 works.

bfendler commented 7 years ago

Yeah, but I was really wanting to get the US codes. This looks pretty deliberate...

bfendler commented 7 years ago

BTW, thanks for this code. Definitely helped me learn something.

Benny- commented 7 years ago

It might be best to use another source for the ticket symbol information instead of yahoo. If anyone got any suggestions please say so.

For now I have put the output of a older run of the program online. you can find it on https://bennyjacobs.nl/market/. This might not be available forever.

bfendler commented 7 years ago

@Benny- you're awesome! Thanks for that. It will help me get started.

Benny- commented 7 years ago

No problem @bfendler , have fun.