Single stock example is broken, unable to fetch Yahoo/Google data

[JVillella]

Running,

https://github.com/quantopian/pyfolio/blob/4b901f6d73aa02ceb6d04b7d83502e5c6f2e81aa/pyfolio/examples/single_stock_example.ipynb

Fails on,

stock_rets = pf.utils.get_symbol_rets('FB')

With,

---------------------------------------------------------------------------
KeyError                                  Traceback (most recent call last)
~/src/portfolio/.venv/lib/python3.8/site-packages/pandas/core/indexes/base.py in get_loc(self, key, method, tolerance)
   2888             try:
-> 2889                 return self._engine.get_loc(casted_key)
   2890             except KeyError as err:

pandas/_libs/index.pyx in pandas._libs.index.IndexEngine.get_loc()

pandas/_libs/index.pyx in pandas._libs.index.IndexEngine.get_loc()

pandas/_libs/hashtable_class_helper.pxi in pandas._libs.hashtable.PyObjectHashTable.get_item()

pandas/_libs/hashtable_class_helper.pxi in pandas._libs.hashtable.PyObjectHashTable.get_item()

KeyError: 'date'

The above exception was the direct cause of the following exception:

KeyError                                  Traceback (most recent call last)
~/src/portfolio/.venv/lib/python3.8/site-packages/empyrical/utils.py in get_symbol_returns_from_yahoo(symbol, start, end)
    435         px = web.get_data_yahoo(symbol, start=start, end=end)
--> 436         px['date'] = pd.to_datetime(px['date'])
    437         px.set_index('date', drop=False, inplace=True)

~/src/portfolio/.venv/lib/python3.8/site-packages/pandas/core/frame.py in __getitem__(self, key)
   2901                 return self._getitem_multilevel(key)
-> 2902             indexer = self.columns.get_loc(key)
   2903             if is_integer(indexer):

~/src/portfolio/.venv/lib/python3.8/site-packages/pandas/core/indexes/base.py in get_loc(self, key, method, tolerance)
   2890             except KeyError as err:
-> 2891                 raise KeyError(key) from err
   2892 

KeyError: 'date'

During handling of the above exception, another exception occurred:

AttributeError                            Traceback (most recent call last)
<ipython-input-7-d8bc8079fb93> in <module>
----> 1 stock_rets = pf.utils.get_symbol_rets('FB')

~/src/portfolio/.venv/lib/python3.8/site-packages/pyfolio/utils.py in get_symbol_rets(symbol, start, end)
    483     """
    484 
--> 485     return SETTINGS['returns_func'](symbol,
    486                                     start=start,
    487                                     end=end)

~/src/portfolio/.venv/lib/python3.8/site-packages/empyrical/deprecate.py in wrapper(*args, **kwargs)
     41                 stacklevel=stacklevel
     42             )
---> 43             return fn(*args, **kwargs)
     44         return wrapper
     45     return deprecated_dec

~/src/portfolio/.venv/lib/python3.8/site-packages/empyrical/utils.py in default_returns_func(symbol, start, end)
    491         rets = rets[start:end]
    492     else:
--> 493         rets = get_symbol_returns_from_yahoo(symbol, start=start, end=end)
    494 
    495     return rets[symbol]

~/src/portfolio/.venv/lib/python3.8/site-packages/empyrical/deprecate.py in wrapper(*args, **kwargs)
     41                 stacklevel=stacklevel
     42             )
---> 43             return fn(*args, **kwargs)
     44         return wrapper
     45     return deprecated_dec

~/src/portfolio/.venv/lib/python3.8/site-packages/empyrical/utils.py in get_symbol_returns_from_yahoo(symbol, start, end)
    441             'Yahoo Finance read failed: {}, falling back to Google'.format(e),
    442             UserWarning)
--> 443         px = web.get_data_google(symbol, start=start, end=end)
    444         rets = px[['Close']].pct_change().dropna()
    445 

AttributeError: module 'pandas_datareader.data' has no attribute 'get_data_google'

---

Versions

• Pyfolio version: 0.9.2
• Python version: 3.8.5
• Pandas version: 1.1.1
• Matplotlib version: 3.3.1

[JVillella]

Looks like google finance API support has been removed from pandas datareader package: https://pydata.github.io/pandas-datareader/stable/index.html (top notice).

[FriendlyUser]

Is there any way to switch the data source to iex or something else?

[jjphung]

Same problem

Versions

Pyfolio version: 0.9.2 Python version: 3.6.10 Pandas version: 1.0.5 Matplotlib version: 3.2.2

[jjphung]

Is there any way to switch the data source to iex or something else?

For now, I manually calculate benchmark returns using a data API of my choice.

[yitelee]

Patch it like this:

from pandas_datareader import data as web
import empyrical
import pyfolio as pf
import pandas as pd
import numpy as np

def get_max_drawdown_underwater_f(underwater):
    """
    Determines peak, valley, and recovery dates given an 'underwater'
    DataFrame.

    An underwater DataFrame is a DataFrame that has precomputed
    rolling drawdown.

    Parameters
    ----------
    underwater : pd.Series
       Underwater returns (rolling drawdown) of a strategy.

    Returns
    -------
    peak : datetime
        The maximum drawdown's peak.
    valley : datetime
        The maximum drawdown's valley.
    recovery : datetime
        The maximum drawdown's recovery.
    """

    #valley = np.argmin(underwater)  # end of the period
    valley = underwater.index[np.argmin(underwater)] # end of the period

    # Find first 0
    peak = underwater[:valley][underwater[:valley] == 0].index[-1]
    # Find last 0
    try:
        recovery = underwater[valley:][underwater[valley:] == 0].index[0]
    except IndexError:
        recovery = np.nan  # drawdown not recovered
    return peak, valley, recovery

def get_symbol_returns_from_yahoo_f(symbol, start=None, end=None):
    """
    Wrapper for pandas.io.data.get_data_yahoo().
    Retrieves prices for symbol from yahoo and computes returns
    based on adjusted closing prices.

    Parameters
    ----------
    symbol : str
        Symbol name to load, e.g. 'SPY'
    start : pandas.Timestamp compatible, optional
        Start date of time period to retrieve
    end : pandas.Timestamp compatible, optional
        End date of time period to retrieve

    Returns
    -------
    pandas.DataFrame
        Returns of symbol in requested period.
    """

    try:
        px = web.get_data_yahoo(symbol, start=start, end=end)
        px['date'] = px.index.to_list()
        #px['date'] = px['date'].apply(lambda x: pd.Timestamp(x))
        #px['date'] = pd.to_datetime(px['date'])
        #px['date'] = pd.to_datetime(px['date'], unit='s')
        px.set_index('date', drop=False, inplace=True)

        #px.index.rename('date',inplace=True)
        rets = px[['Adj Close']].pct_change().dropna()
        rets.rename(columns={"Adj Close": "adjclose"},inplace=True)
    except Exception as e:
        warnings.warn(
            'Yahoo Finance read failed: {}, falling back to Google'.format(e),
            UserWarning)
        px = web.get_data_google(symbol, start=start, end=end)
        rets = px[['Close']].pct_change().dropna()

    rets.index = rets.index.tz_localize("UTC")
    rets.columns = [symbol]
    return rets

empyrical.utils.get_symbol_returns_from_yahoo = get_symbol_returns_from_yahoo_f
pf.timeseries.get_max_drawdown_underwater = get_max_drawdown_underwater_f

[DimaDDM]

Please fix this issue. Library totally useless!

[aidiss]

This looks like a great library, a lot of work was done here. It is so sad that it is completely abandoned.

[hemangjoshi37a]

This looks like a great library, a lot of work was done here. It is so sad that it is completely abandoned.

Truly said bro... Please if anyone can fix and merge... https://hjlabs.in/

[hemangjoshi37a]

I have got it working a bit still it does not count the real answers to all the output values. Please someone help.

single_stock_example.ipynb.zip https://hjlabs.in/

[Photon1c]

seems the issue is datareader source google is called for me, when I want it to call the working yahoo source instead. Quantopian also seems to have gone off the grid :/

[bommarito18]

Patch it like this:

from pandas_datareader import data as web
import empyrical
import pyfolio as pf
import pandas as pd
import numpy as np

def get_max_drawdown_underwater_f(underwater):
    """
    Determines peak, valley, and recovery dates given an 'underwater'
    DataFrame.

    An underwater DataFrame is a DataFrame that has precomputed
    rolling drawdown.

    Parameters
    ----------
    underwater : pd.Series
       Underwater returns (rolling drawdown) of a strategy.

    Returns
    -------
    peak : datetime
        The maximum drawdown's peak.
    valley : datetime
        The maximum drawdown's valley.
    recovery : datetime
        The maximum drawdown's recovery.
    """

    #valley = np.argmin(underwater)  # end of the period
    valley = underwater.index[np.argmin(underwater)] # end of the period

    # Find first 0
    peak = underwater[:valley][underwater[:valley] == 0].index[-1]
    # Find last 0
    try:
        recovery = underwater[valley:][underwater[valley:] == 0].index[0]
    except IndexError:
        recovery = np.nan  # drawdown not recovered
    return peak, valley, recovery

def get_symbol_returns_from_yahoo_f(symbol, start=None, end=None):
    """
    Wrapper for pandas.io.data.get_data_yahoo().
    Retrieves prices for symbol from yahoo and computes returns
    based on adjusted closing prices.

    Parameters
    ----------
    symbol : str
        Symbol name to load, e.g. 'SPY'
    start : pandas.Timestamp compatible, optional
        Start date of time period to retrieve
    end : pandas.Timestamp compatible, optional
        End date of time period to retrieve

    Returns
    -------
    pandas.DataFrame
        Returns of symbol in requested period.
    """

    try:
        px = web.get_data_yahoo(symbol, start=start, end=end)
        px['date'] = px.index.to_list()
        #px['date'] = px['date'].apply(lambda x: pd.Timestamp(x))
        #px['date'] = pd.to_datetime(px['date'])
        #px['date'] = pd.to_datetime(px['date'], unit='s')
        px.set_index('date', drop=False, inplace=True)

        #px.index.rename('date',inplace=True)
        rets = px[['Adj Close']].pct_change().dropna()
        rets.rename(columns={"Adj Close": "adjclose"},inplace=True)
    except Exception as e:
        warnings.warn(
            'Yahoo Finance read failed: {}, falling back to Google'.format(e),
            UserWarning)
        px = web.get_data_google(symbol, start=start, end=end)
        rets = px[['Close']].pct_change().dropna()

    rets.index = rets.index.tz_localize("UTC")
    rets.columns = [symbol]
    return rets

empyrical.utils.get_symbol_returns_from_yahoo = get_symbol_returns_from_yahoo_f
pf.timeseries.get_max_drawdown_underwater = get_max_drawdown_underwater_f

This is awesome, Thank you! Anyone knows how to add dates to the charts?

[jjphung]

Hey ya'll! Try out: https://github.com/stefan-jansen/pyfolio-reloaded. Looks like a solid, updated fork.

[yitelee]

This is the decent successor, https://github.com/ranaroussi/quantstats