Wrong number of symbols returned

[dcottle]

I installed this into an Anaconda python distribution in Linux.

I ran sp500 = finsymbols.get_sp500_symbols() in a terminal.

The list returned from the command had length of only 341. Same result after running this several times.

The last stock symbols in the list are NU, NRG, NUE, and NVDA.

Any ideas how to correct this?

[skillachie]

What version are you using ? Make sure you are using https://pypi.python.org/pypi/finsymbols

And are you sure you ran it correctly ? All tests currently pass. Share your script so I can better help to debug

[skillachie]

Created an example gist for you https://gist.github.com/skillachie/4b4dbdf5e862e1cfa3b9 . 502 symbols will be returned because of stock splits

Seem like there is an issue in your script

[dcottle]

I installed finsymbols today with

pip install finsymbols

The package installed without error.

I have only tried using this in ipython, and I used

import finsymbols

sp500= finsymbols.get_sp500_symbols()

and also

from finsymbols import get_sp500_symbols

sp500 = get_sp500_symbols()

Same result with the length of the sp500 list as 341.

I also tried your recommendation of printing the items in the list individually. The symbols printed up to NVDA.

It seems like your code works fine, but it doesn’t finish. Could it be due to an error in the Wikipedia page that you are scraping to extract the data?

Is there anything else I can try? I really like what you are doing to extract the SP500 list. I thought about doing this myself, then found your solution which is certainly better than what I could have come up with.

Thanks for your efforts.

Sent from Windows Mail

From: Dwayne Campbell Sent: ‎Monday‎, ‎February‎ ‎2‎, ‎2015 ‎4‎:‎37‎ ‎PM To: skillachie/finsymbols Cc: dave.cottle@gmail.com

Created an example gist for you https://gist.github.com/skillachie/4b4dbdf5e862e1cfa3b9 . 502 symbols will be returned because of stock splits

Seem like there is an issue in your script

— Reply to this email directly or view it on GitHub.

[skillachie]

Wish I could help, but I cannot simulate the issue your having. The module currently gets more than 400 installs each month with no issues and have had an additional contributor that have verified the functionality.

Its verified to be working on all *nix systems.

I am honestly not sure what else to tell you to try. I just tested on a new environment using virtualenv https://pypi.python.org/pypi/virtualenv

Use regular python instead of Anaconda python distribution to see if your results change. You could also share your script for me to take a quick look to see how your count

[dcottle]

The complete script that I used to test the package is:

import finsymbols

sp500 = finsymbols.get_sp500_symbols()

len(sp500)

which gives 341 as the length of sp500. Also

sp500[-1][‘symbol’] gives NVDA.

I also tried to get the nyse symbols. Is 3301 the correct number of securities on the NYSE?

Sent from Windows Mail

From: Dwayne Campbell Sent: ‎Monday‎, ‎February‎ ‎2‎, ‎2015 ‎6‎:‎45‎ ‎PM To: skillachie/finsymbols Cc: dave.cottle@gmail.com

Wish I could help, but I cannot simulate the issue your having. The module currently gets more than 400 installs each month with no issues and have had an additional contributor that have verified the functionality.

Its verified to be working on all *nix systems.

screen shot 2015-02-02 at 6 35 33 pm

I am honestly not sure what else to tell you to try. I just tested on a new environment using virtualenv https://pypi.python.org/pypi/virtualenv

Use regular python instead of Anaconda python distribution to see if your results change. You could also share your script for me to take a quick look to see how your count

— Reply to this email directly or view it on GitHub.

[skillachie]

Dave,

Cool. Did the exact same thing and still cannot repeat the issue you are having unfortunately. The only suggestions I would have is to try regular Python or parse the Wiki page. Without knowing exactly why it fails in your environment I cannot assist

The only thing I can think about is that your internet connection might have been interrupted when you tried to fetch the SP500 list initially and so has only a partial list which would be very unlikely . But this might be cached here python_env/lib/python2.7/site-packages/finsymbols-0.5.0-py2.7.egg/finsymbols/SP500.html

Check to see if symbols ZTS and others are present in the cached html. This cache is refreshed every 24 hours. If the symbol is not present in the cached html, delete it.

If all symbols are present in the raw html file. There is something specific to your environment which might be due to Anaconda python distribution. Other than that I cannot help. All the best.

[skillachie]

Screenshot with 1.0.0

[dcottle]

I also installed finsymbols in Windows 8.1, also using the Anaconda python distribution.

This time, the number of stock symbols is 502, and the last stock is ZTS, so this worked.

I did delete the SP500.html file in the egg directory on my Linux computer, but that didn’t seem to help.

It seems as if finsymbols was installed incorrectly, or there is a problem with Anaconda on my Linux computer.

How would you reinstall finsymbols?

Sent from Windows Mail

From: Dwayne Campbell Sent: ‎Monday‎, ‎February‎ ‎2‎, ‎2015 ‎9‎:‎29‎ ‎PM To: skillachie/finsymbols Cc: dave.cottle@gmail.com

Dave,

Cool. Did the exact same thing and still cannot repeat the issue you are having unfortunately. The only suggestions I would have is to try regular Python or parse the Wiki page. Without knowing exactly why it fails in your environment.

The only thing I can think about is that your internet connection might have been interrupted when you tried to fetch the SP500 list initially and so has only a partial list which would be very unlikely . But this might be catch here python_env/lib/python2.7/site-packages/finsymbols-0.5.0-py2.7.egg/finsymbols/SP500.html

Check to see if symbols ZTS and others are present in the cached html. This cache is refreshed every 24 hours. If all symbols are present in the raw html file. There is something specific to your environment which might be due to Anaconda python distribution. Other than that I cannot help. All the best.

screen shot 2015-02-02 at 9 20 09 pm

— Reply to this email directly or view it on GitHub.

[skillachie]

Glad to know it worked in your other environment. There might be something that is further wrong in your other environment but you can remove it by doing

pip uninstall finsymbols

As good measure you could check to ensure all files are removed from your python site-packages folder

[skillachie]

Going to close this since you got it working in another environment. Update with your comment if you figure out what is wrong with your other environment

[dcottle]

I fixed the problem for Linux (Fedora 21 in this case).

I added the parser html5lib to the BeautifulSoup call:

 wiki_soup = BeautifulSoup(page_html, 'html5lib')

Seems to work great now. Thanks for your help.

[skillachie]

Thank you for keeping me updated. That is good to know. It currently works with all other OS I might make this change after I do some testing.

Fedora 21 might give you unnecessary pain when doing development work.

Happy coding.