search

Benny- / Yahoo-ticker-symbol-downloader

A web scraper for ticker symbols from yahoo finance
https://pypi.python.org/pypi/Yahoo-ticker-downloader/
Other
247 stars 87 forks source link

Does not pull symbols starting with the letter "A" #54

Open NLockwood opened 5 years ago

NLockwood commented 5 years ago

Is this a limitation set by Yahoo? If so, is there a work-around? Thanks!

Bruno-Hanzen commented 4 years ago

It's an error in the recursion algorithm. The algorithm works like this: -initialise search strings with this: "first_search_characters = 'abcdefghijklmnopqrstuvwxyz'" -start recursion with "nextQuery", which start with adding 1 to the index. Python starts indexing at 0, so algorithm starts at index 1, which is letter "b". -when everything is exhausted, do "a" when all the rest has been exhausted, but the condition is never fullfilled. -when everything +a is exhausted, decode the pickle and write results.

I have modified this by: -"first_search_characters = ' abcdefghijklmnopqrstuvwxyz'" (insert a space before the first character -when this is done, it is also possible to simplify the testing of the exit condition.

My code is running since a few days now; When it is completed, i'll send a pull request with my changes.

AquibaB commented 3 years ago

Hi Bruno, Thanks for the good work here! Did you get a positive outcome from your test? Thanks! Aquiba

Bruno-Hanzen commented 3 years ago

Hi Aquiba,

sorry, forgot to post my results. Yes, it works. And I have posted a PR - no feedback yet

ProlorenzoAndrii commented 3 years ago

Hi Aquiba,

sorry, forgot to post my results. Yes, it works. And I have posted a PR - no feedback yet

Hi sir, Can you share somewhere your result file?

Bruno-Hanzen commented 3 years ago

Hi all,

several people have asked me to share my results. I don't do that due to Yahoo terms of use:

https://www.verizonmedia.com/policies/us/en/verizonmedia/terms/otos/index.html

"Unless you have explicit written permission, you must not reproduce, modify, rent, lease, sell, trade, distribute, transmit, broadcast, publicly perform, create derivative works based on, or exploit for any commercial purposes, any portion or use of, or access to, the Services (including content, advertisements, APIs, and software)."

I certainly do not want to get a letter from their lawyers and risk, for the least, to be bounced off their finance site. I still need them to get the daily prices. But you can run the script (with my modifs) and, with some patience (2 months...), you'll have a list of symbols.

thatkotov commented 3 years ago

It's 2021, there's no 'abcd...' line in code & I still can't find symbols starting with A in outputs. No clue how to change the code.