Can't download the ticker list

[danielfmva]

Hi,

I am receiveing the following error when I try to get the ticker symbols data:

╭─dmva@Daniels-MacBook-Pro.local ~  
╰─➤  YahooTickerDownloader.py stocks                                                                        1 ↵
Checking if we can resume a old download session
Downloader found on disk, resuming
Downloading Stock

req https://finance.yahoo.com/lookup/?s=a&r=&m=ALL&t=S&b=0
A exception occurred while downloading. Suspending downloader to disk
Successfully saved download state
Remove downloader.pickle if this error persists
Issues can be reported on https://github.com/Benny-/Yahoo-ticker-symbol-downloader/issues

Traceback (most recent call last):
  File "/usr/local/bin/YahooTickerDownloader.py", line 146, in <module>
    main()
  File "/usr/local/bin/YahooTickerDownloader.py", line 93, in main
    downloadEverything(downloader)
  File "/usr/local/bin/YahooTickerDownloader.py", line 51, in downloadEverything
    symbols = downloader.nextRequest()
  File "/usr/local/lib/python2.7/site-packages/ytd/SymbolDownloader.py", line 73, in nextRequest
    html = self._fetchHtml()
  File "/usr/local/lib/python2.7/site-packages/ytd/SymbolDownloader.py", line 36, in _fetchHtml
    resp = self.rsession.send(req)
  File "/usr/local/lib/python2.7/site-packages/requests/sessions.py", line 573, in send
    r = adapter.send(request, **kwargs)
  File "/usr/local/lib/python2.7/site-packages/requests/adapters.py", line 431, in send
    raise SSLError(e, request=request)
requests.exceptions.SSLError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:581)

Any idea of what is going on?

[Benny-]

I could not reproduce this exception.

The exception you have pasted still contains some information to point to a possible cause.

YahooTickerDownloader.py tries to connect to yahoo finance using a secure connection. But this fails. It is possible you do not have the proper root certificate in your trust store. Another possible problem is YahooTickerDownloader.py does not use the proper system trust store. I can not help you check which of those problems it is.

Fixing the trust store issue might be the fastest way. If you would like to wait a few weeks I will be able to add a command line switch in YahooTickerDownloader.py to allow insecure connection to yahoo finance. Are you prepared to wait a while? I might add this anyway if more people people have a problem with a secure SSL/TLS connection.

Please respond if you have fixed this issue yourself.

[danielfmva]

Hi, thank you for the answer!

How do I check if I have or not the proper root certificate? I am trying to run this script on Mac OS... I think It may be related. Is this script system dependant?

[Benny-]

There is no fundamental reason why it should not work on a Mac OS. Can you tell me the way you have installed the YahooTickerDownloader.

[danielfmva]

Thank for answering! I have installed through pip install command.

[Benny-]

Yeah... Searching for the underlying cause will take a lot of effort. I have opted for another solution. There is a new YahooTickerDownloader version on github. This new version allows insecure connections using the -i switch. This should sidestep your problem.

The obvious downside is the use of a insecure connection. This allows organizations like the NSA to spy on us.

It will take some test runs before I upload it to the python packaging index so you should download/clone it from github.

If you ever find the original cause, please add it to this ticket as a comment.

[Benny-]

@danielfmva, can you check if you can visit https://finance.yahoo.com/lookup/ in safari web browser?

You have the proper root certificate if the website is shown without any security alert.

[knoguchi]

It's a Python in Mac OSX problem that linked against old OpenSSL.

[Benny-]

Thanks @knoguchi. I will close this issue.