search

Benny- / Yahoo-ticker-symbol-downloader

A web scraper for ticker symbols from yahoo finance
https://pypi.python.org/pypi/Yahoo-ticker-downloader/
Other
247 stars 87 forks source link

Filter multiple exhanges #50

Open chicoriagithub opened 5 years ago

chicoriagithub commented 5 years ago

it is not working for spanish tickets

Yahoo ticker dowloader.py stocks -m ES no working Yahoo ticker dowloader.py stocks -m es no working

Yahoo ticker dowloader.py stocks -E anything work it is a bug Yahoo ticker dowloader.py stocks -E es or -E NYQ

Coudl you suggest me any alternative to take only a market o several markets o exchange

Benny- commented 5 years ago

I recommend to download everything and to do the filtering after you downloaded everything. Would that suffice? Alternatively I can add a feature request for filtering multiple exchanges. Filtering by country or region is very difficult as yahoo doesn't return the proper results.

chicoriagithub commented 5 years ago

ok it would be very nice to be a request filtering multiple exchange, we wiil save a lot of time, for example MCE(spain) SSH(Sangai),etc.. thanks a lot for your support

Benny- commented 5 years ago

Okay, do mind that there is no ETA right now and you are able to do this using tools like excel.

chicoriagithub commented 5 years ago

yes but the problem is excel too many datas to move all , in the previus version you can filter in the first request, how can we do it? thanks

Benny- commented 5 years ago

Have you also tried libreoffice?

Do note that filtering within Yahoo-ticker-symbol-downloader is always done after all requests. To make it happen it should accept multiple exchange parameters and filter them here.

chicoriagithub commented 5 years ago

YahooTickerDownloader.py -E NYQ it is correct only for newyork exchange?

chicoriagithub commented 5 years ago

optional arguments: -h, --help show this help message and exit -i, --insecure use HTTP instead of HTTPS -e, --export export immediately without downloading (Only useful if you already downloaded something to the .pickle file) -E EXCHANGE, --Exchange EXCHANGE Only export ticker symbols from this exchange (the filtering is done during the export phase) -s SLEEP, --sleep SLEEP The time to sleep in seconds between requests -p, --pandantic Stop and warn the user if some rare assertion fails -m MARKET, --market MARKET Specify the Region of queried exchanges (us = USA+Canada, dr=Germany, fr=France, hk=Hongkong, gb=United Kingdom, default= all)

it was in the old version and you can reduce the time for dowloading working with markets

chicoriagithub commented 5 years ago

if you have one example give us please to know how can dowload the tickets with less time thanks

Benny- commented 5 years ago

The exchange filtering is done once everything is downloaded. So this feature will not take less time for the script to run.

The market parameter you described in your example would not work as there is no way for the script to map exchanges to countries.

The "Filter multiple exchanges" feature is not implemented so I can not give any examples.

chicoriagithub commented 5 years ago

ok thanks in advance

After execute the system had this messages error:

Saving downloader to disk... Downloader successfully saved.

req https://finance.yahoo.com/_finance_doubledown/api/resource/searchassist;searchTerm=null?device=console&returnMeta=true Retry attempt: 1 of 5. Sleep period: 5 seconds. req https://finance.yahoo.com/_finance_doubledown/api/resource/searchassist;searchTerm=null?device=console&returnMeta=true Retry attempt: 2 of 5. Sleep period: 25 seconds. req https://finance.yahoo.com/_finance_doubledown/api/resource/searchassist;searchTerm=null?device=console&returnMeta=true Retry attempt: 3 of 5. Sleep period: 125 seconds. req https://finance.yahoo.com/_finance_doubledown/api/resource/searchassist;searchTerm=null?device=console&returnMeta=true Retry attempt: 4 of 5. Sleep period: 625 seconds. req https://finance.yahoo.com/_finance_doubledown/api/resource/searchassist;searchTerm=null?device=console&returnMeta=true Retry attempt: 5 of 5. Sleep period: 3125 seconds. req https://finance.yahoo.com/_finance_doubledown/api/resource/searchassist;searchTerm=null?device=console&returnMeta=true A exception occurred while downloading. Suspending downloader to disk Successfully saved download state Try removing {type}.pickle file if this error persists Issues can be reported on https://github.com/Benny-/Yahoo-ticker-symbol-downloader/issues

Traceback (most recent call last): File "/home/mountainers/.local/bin/YahooTickerDownloader.py", line 170, in main() File "/home/mountainers/.local/bin/YahooTickerDownloader.py", line 111, in main downloadEverything(downloader, tickerType, args.insecure, args.sleep, args.pandantic) File "/home/mountainers/.local/bin/YahooTickerDownloader.py", line 40, in downloadEverything symbols = downloader.nextRequest(insecure, pandantic) File "/home/mountainers/.local/lib/python3.6/site-packages/ytd/SimpleSymbolDownloader.py", line 95, in nextRequest json = self._fetch(insecure) File "/home/mountainers/.local/lib/python3.6/site-packages/ytd/SimpleSymbolDownloader.py", line 65, in _fetch resp.raise_for_status() File "/home/mountainers/.local/lib/python3.6/site-packages/requests/models.py", line 940, in raise_for_status raise HTTPError(http_error_msg, response=self) requests.exceptions.HTTPError: 503 Server Error: Service Unavailable for url: https://finance.yahoo.com/_finance_doubledown/api/resource/searchassist;searchTerm=null?device=console&returnMeta=true

1) Whats the name of the file dowloaded?

Thanks very much for your support

Benny- commented 5 years ago

The output of the program will be "generic.csv" among other files.

I created another issue for that new problem.

Benny- commented 5 years ago

Can you answer the question I asked there?