Download load data - Exception handling

[andrewczgithub]

Hi @ranaroussi I am trying to download all the sp500 data using the below script, however i am running into trouble with the exception handling-

from pandas_datareader import data as pdr

import yfinance as yf
yf.pdr_override() # <== that's all it takes :-)

# NYSE
url_nyse = "http://www.nasdaq.com/screening/companies-by-name.aspx?letter=0&exchange=nyse&render=download"
# Nasdaq
url_nasdaq = "http://www.nasdaq.com/screening/companies-by-name.aspx?letter=0&exchange=nasdaq&render=download"
# AMEX
url_amex = "http://www.nasdaq.com/screening/companies-by-name.aspx?letter=0&exchange=amex&render=download"

import pandas as pd

df = pd.DataFrame.from_csv(url_nyse)
stocks = df.index.tolist()

df_list = []
used_stocks = []

for stock in stocks:
    try:
        data = pdr.get_data_yahoo(stock, start="2017-01-01", end="2017-04-30")
        data_close=data['Close']
        df_list.append(data_close)
        used_stocks.append(stock)
    except (KeyError, ValueError):
        print("Error")

df_list

It comes up with a traceback error. Best, Andrew

[ranaroussi]

Running your code and seeing what's in stocks, I get [0, 1, 2, ...]. I think what you need to do is replace stocks = df.index.tolist() with stocks = df['Symbol'].tolist().

Your code can also be much simpler... Try this:

import pandas as pd
from pandas_datareader import data as pdr
import yfinance as yf
yf.pdr_override()

url_nyse = "http://www.nasdaq.com/screening/companies-by-name.aspx?letter=0&exchange=nyse&render=download"

df = pd.read_csv(url_nyse)
stocks = df['Symbol'].tolist()

data = pdr.get_data_yahoo(stocks, start="2017-01-01", end="2017-04-30")

data.head()

[ranaroussi]

Closing for now. Please reopen issue if the problem persists.