The excel file no longer contains a list of constituents

datasets / s-and-p-500-companies

List of companies in the S&P 500 together with associated financials

https://datahub.io/core/s-and-p-500-companies

499 stars 491 forks source link

The excel file no longer contains a list of constituents #8

Closed chafreaky closed 8 years ago

chafreaky commented 9 years ago

Hey there -

I am not sure how the excel file at this address was structured before, but as of today, it no longer contains a list of S&P500 constituents. Right now, sheet 4 contains a list of companies that issued additional shares in Q2 of 2015.

Am I missing something?

Thanks, Chafik

lexman commented 8 years ago

You're right ! For example, the sheet doesn't mention Alphabet Inc'A' shares.

After searching a proper source for this datapackage, I've ended scraping wikipedia's list (https://en.wikipedia.org/wiki/List_of_S%26P_500_companies) for two reasons :

the page is very well maintained, with two regular contributors updating the list for the past two years, plus several irregular contributor
licence is cristal clear, wich is not be the case with financial website/TV like quandl, yahoo, CNN...

The code is available on my github https://github.com/lexman/s-and-p-500-companies (on branch tuttle)

@rgrp @pdehaye here are the links :

rufuspollock commented 8 years ago

@lexman this is great and I agree Wikipedia may now be a better bet (though it would be nice to know where the S&P file has gone).

A couple of minor suggestions:

Could you submit as a Pull Request onto master here
Could we get a makefile. We are increasingly of the view that we should default to Makefiles for building (even if this then calls Tuttle). Also, in this case could you summarize the difference between tuttle and simple python option -- in general, here i would have an inclination to keeping this simple.
Also the tuttle branch seems to have a lot of commits (13k or so!). I assume this is data being updated. Do we want this in the initial PR? Not sure we do but open to discussion (and i assume this is daily data - because prices etc change every day -- i am starting to think if we go this route we should separate s&p500 companies and their data into separate repos maybe but that is a bigger discussion).

lexman commented 8 years ago

Woups, too much precipitation, I'm not sure you want all my tests... I'll make a proper branch (with a makefile) and make a pull request.

About makefiles, if there is a reflexion ongoing I might have some suggestions. Where can I speak about my experience with a lot of data at http://mappy.com ? Maybe in the lab ?

rufuspollock commented 8 years ago

@lexman brilliant - once your PR is ready, great to look at it.

In terms of general discussion re makefiles etc i suggest keeping posting in the forum https://discuss.okfn.org/c/open-knowledge-labs - which I know you have already been doing :-) e.g. https://discuss.okfn.org/t/a-tool-for-collaborating-on-datapackages/1397

lexman commented 8 years ago

Closed a while ago...