search

robertmartin8 / PyPortfolioOpt

Financial portfolio optimisation in python, including classical efficient frontier, Black-Litterman, Hierarchical Risk Parity
https://pyportfolioopt.readthedocs.io/
MIT License
4.39k stars 940 forks source link

pandas version for "@" operator between DataFrame and Series #59

Closed schneiderfelipe closed 4 years ago

schneiderfelipe commented 4 years ago

I just tested the latest repo version of PyPortfolioOpt and got some errors related to the usage of @: TypeError: unsupported operand type(s) for @: 'DataFrame' and 'Series'.

I installed PyPortfolioOpt with pip3 -e ., which means all dependencies were satisfied.

Pandas stable (0.25.3) says @ can be used for dot product. But since I'm using Pandas 0.25.1, it might be a version issue.

Since PyPortfolioOpt currently depends on Pandas 0.21.0 or higher, should we change requirements.txt?

Full test output:

➜  PyPortfolioOpt git:(master) ✗ pytest-3
=============================================================================================== test session starts ===============================================================================================
platform linux -- Python 3.6.9, pytest-3.3.2, py-1.5.2, pluggy-0.6.0
rootdir: /home/schneider/Dropbox/PyPortfolioOpt, inifile:
collected 140 items                                                                                                                                                                                               

tests/test_base_optimizer.py ............                                                                                                                                                                   [  8%]
tests/test_black_litterman.py ..........FF                                                                                                                                                                  [ 17%]
tests/test_cla.py ..........                                                                                                                                                                                [ 24%]
tests/test_custom_objectives.py ......                                                                                                                                                                      [ 28%]
tests/test_discrete_allocation.py ..............                                                                                                                                                            [ 38%]
tests/test_efficient_frontier.py .........................................                                                                                                                                  [ 67%]
tests/test_expected_returns.py .........                                                                                                                                                                    [ 74%]
tests/test_hrp.py .....                                                                                                                                                                                     [ 77%]
tests/test_objective_functions.py .......                                                                                                                                                                   [ 82%]
tests/test_risk_models.py ...................                                                                                                                                                               [ 96%]
tests/test_value_at_risk.py .....                                                                                                                                                                           [100%]

==================================================================================================== FAILURES =====================================================================================================
____________________________________________________________________________________________ test_market_implied_prior ____________________________________________________________________________________________

    def test_market_implied_prior():
        df = get_data()
        S = risk_models.sample_cov(df)

        prices = pd.read_csv(
            "tests/spy_prices.csv", parse_dates=True, index_col=0, squeeze=True
        )
        delta = black_litterman.market_implied_risk_aversion(prices)

        mcaps = {
            "GOOG": 927e9,
            "AAPL": 1.19e12,
            "FB": 574e9,
            "BABA": 533e9,
            "AMZN": 867e9,
            "GE": 96e9,
            "AMD": 43e9,
            "WMT": 339e9,
            "BAC": 301e9,
            "GM": 51e9,
            "T": 61e9,
            "UAA": 78e9,
            "SHLD": 0,
            "XOM": 295e9,
            "RRC": 1e9,
            "BBY": 22e9,
            "MA": 288e9,
            "PFE": 212e9,
            "JPM": 422e9,
            "SBUX": 102e9,
        }
>       pi = black_litterman.market_implied_prior_returns(mcaps, delta, S)

tests/test_black_litterman.py:281: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

market_caps = {'AAPL': 1190000000000.0, 'AMD': 43000000000.0, 'AMZN': 867000000000.0, 'BABA': 533000000000.0, ...}, risk_aversion = 2.6854910662283147
cov_matrix =           GOOG      AAPL        FB      BABA      AMZN        GE       AMD  \
GOOG  0.093211  0.046202  0.030801  0.02...  0.056298  0.070269  0.034757  0.146893  0.049530  
SBUX  0.028284  0.050217  0.046886  0.024195  0.049530  0.152589  
risk_free_rate = 0.02

    def market_implied_prior_returns(
        market_caps, risk_aversion, cov_matrix, risk_free_rate=0.02
    ):
        r"""
        Compute the prior estimate of returns implied by the market weights.
        In other words, given each asset's contribution to the risk of the market
        portfolio, how much are we expecting to be compensated?

        .. math::

            \Pi = \delta \Sigma w_{mkt}

        :param market_caps: market capitalisations of all assets
        :type market_caps: {ticker: cap} dict or pd.Series
        :param risk_aversion: risk aversion parameter
        :type risk_aversion: positive float
        :param cov_matrix: covariance matrix of asset returns
        :type cov_matrix: pd.DataFrame or np.ndarray
        :param risk_free_rate: risk-free rate of borrowing/lending, defaults to 0.02.
                               You should use the appropriate time period, corresponding
                               to the covariance matrix.
        :type risk_free_rate: float, optional
        :return: prior estimate of returns as implied by the market caps
        :rtype: pd.Series
        """
        mcaps = pd.Series(market_caps)
        mkt_weights = mcaps / mcaps.sum()
        # Pi is excess returns so must add risk_free_rate to get return.
>       return risk_aversion * cov_matrix @ mkt_weights + risk_free_rate
E       TypeError: unsupported operand type(s) for @: 'DataFrame' and 'Series'

pypfopt/black_litterman.py:44: TypeError
________________________________________________________________________________________ test_black_litterman_market_prior ________________________________________________________________________________________

    def test_black_litterman_market_prior():
        df = get_data()
        S = risk_models.sample_cov(df)

        prices = pd.read_csv(
            "tests/spy_prices.csv", parse_dates=True, index_col=0, squeeze=True
        )
        delta = black_litterman.market_implied_risk_aversion(prices)

        mcaps = {
            "GOOG": 927e9,
            "AAPL": 1.19e12,
            "FB": 574e9,
            "BABA": 533e9,
            "AMZN": 867e9,
            "GE": 96e9,
            "AMD": 43e9,
            "WMT": 339e9,
            "BAC": 301e9,
            "GM": 51e9,
            "T": 61e9,
            "UAA": 78e9,
            "SHLD": 0,
            "XOM": 295e9,
            "RRC": 1e9,
            "BBY": 22e9,
            "MA": 288e9,
            "PFE": 212e9,
            "JPM": 422e9,
            "SBUX": 102e9,
        }
>       prior = black_litterman.market_implied_prior_returns(mcaps, delta, S)

tests/test_black_litterman.py:351: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

market_caps = {'AAPL': 1190000000000.0, 'AMD': 43000000000.0, 'AMZN': 867000000000.0, 'BABA': 533000000000.0, ...}, risk_aversion = 2.6854910662283147
cov_matrix =           GOOG      AAPL        FB      BABA      AMZN        GE       AMD  \
GOOG  0.093211  0.046202  0.030801  0.02...  0.056298  0.070269  0.034757  0.146893  0.049530  
SBUX  0.028284  0.050217  0.046886  0.024195  0.049530  0.152589  
risk_free_rate = 0.02

    def market_implied_prior_returns(
        market_caps, risk_aversion, cov_matrix, risk_free_rate=0.02
    ):
        r"""
        Compute the prior estimate of returns implied by the market weights.
        In other words, given each asset's contribution to the risk of the market
        portfolio, how much are we expecting to be compensated?

        .. math::

            \Pi = \delta \Sigma w_{mkt}

        :param market_caps: market capitalisations of all assets
        :type market_caps: {ticker: cap} dict or pd.Series
        :param risk_aversion: risk aversion parameter
        :type risk_aversion: positive float
        :param cov_matrix: covariance matrix of asset returns
        :type cov_matrix: pd.DataFrame or np.ndarray
        :param risk_free_rate: risk-free rate of borrowing/lending, defaults to 0.02.
                               You should use the appropriate time period, corresponding
                               to the covariance matrix.
        :type risk_free_rate: float, optional
        :return: prior estimate of returns as implied by the market caps
        :rtype: pd.Series
        """
        mcaps = pd.Series(market_caps)
        mkt_weights = mcaps / mcaps.sum()
        # Pi is excess returns so must add risk_free_rate to get return.
>       return risk_aversion * cov_matrix @ mkt_weights + risk_free_rate
E       TypeError: unsupported operand type(s) for @: 'DataFrame' and 'Series'

pypfopt/black_litterman.py:44: TypeError
================================================================================================ warnings summary =================================================================================================
tests/test_black_litterman.py::test_input_errors
  /home/schneider/Dropbox/PyPortfolioOpt/pypfopt/black_litterman.py:173: UserWarning: Running Black-Litterman with no prior.
    warnings.warn("Running Black-Litterman with no prior.")

tests/test_black_litterman.py::test_parse_views
  /home/schneider/Dropbox/PyPortfolioOpt/pypfopt/black_litterman.py:173: UserWarning: Running Black-Litterman with no prior.
    warnings.warn("Running Black-Litterman with no prior.")

tests/test_black_litterman.py::test_dataframe_input
  /home/schneider/Dropbox/PyPortfolioOpt/pypfopt/black_litterman.py:173: UserWarning: Running Black-Litterman with no prior.
    warnings.warn("Running Black-Litterman with no prior.")

tests/test_black_litterman.py::test_default_omega
  /home/schneider/Dropbox/PyPortfolioOpt/pypfopt/black_litterman.py:173: UserWarning: Running Black-Litterman with no prior.
    warnings.warn("Running Black-Litterman with no prior.")

tests/test_black_litterman.py::test_bl_returns_no_prior
  /home/schneider/Dropbox/PyPortfolioOpt/pypfopt/black_litterman.py:173: UserWarning: Running Black-Litterman with no prior.
    warnings.warn("Running Black-Litterman with no prior.")

tests/test_black_litterman.py::test_bl_relative_views
  /home/schneider/Dropbox/PyPortfolioOpt/pypfopt/black_litterman.py:173: UserWarning: Running Black-Litterman with no prior.
    warnings.warn("Running Black-Litterman with no prior.")

tests/test_black_litterman.py::test_bl_cov_default
  /home/schneider/Dropbox/PyPortfolioOpt/pypfopt/black_litterman.py:173: UserWarning: Running Black-Litterman with no prior.
    warnings.warn("Running Black-Litterman with no prior.")

tests/test_black_litterman.py::test_bl_weights
  /home/schneider/Dropbox/PyPortfolioOpt/pypfopt/black_litterman.py:173: UserWarning: Running Black-Litterman with no prior.
    warnings.warn("Running Black-Litterman with no prior.")

-- Docs: http://doc.pytest.org/en/latest/warnings.html
================================================================================ 2 failed, 138 passed, 8 warnings in 19.60 seconds ================================================================================
robertmartin8 commented 4 years ago

@schneiderfelipe

That's a great spot. Unfortunately, I always forget to update requirements.txt as I am using poetry for version control.

For this particular issue, I will revert it back to .dot since the marginal elegance of using @ is not worth making people update pandas. Will push a fix shortly.