Actual portfolio weight exceeds target percentage when using `PortfolioTarget.Percent`

ArthurAsenheimer commented 2 years ago

Expected Behavior

The portfolio weight is less or equal to the target percentage at the time of order fill.

Actual Behavior

The portfolio weight far exceeds the target percentage when using PortfolioTarget.Percent.

Potential Solution

I think this happens since LEAN uses security.Price to determine the initial margin requirements in line 230 of the buying power model which affects the result of PortfolioTarget.Percent via line 269 in the position group buying power model.

Instead we should use security.AskPrice if target.Quantity > security.Holdings.Quantity (i.e. delta > 0) and otherwise security.BidPrice; at least if quote bars are available for the given security and we therefore can assume to get a fill at the ask/bid price. It also explains why the discrepancy is proportional to the bid-ask spread.

Reproducing the Problem

Run this backtest and view the order history and log file. It is intended to buy 10% of a QQQ option contract, but instead we get 14% which is a huge discrepancy.

from AlgorithmImports import *

class DemoAlgoPortfolioTargetPercentBug(QCAlgorithm):

    def Initialize(self):
        self.SetStartDate(2020, 3, 15)  
        self.SetEndDate(2020, 4, 1)
        self.SetCash(1_000_000)  
        self.UniverseSettings.DataNormalizationMode = DataNormalizationMode.Raw 
        self.Settings.FreePortfolioValuePercentage = 0 
        self.targetPercent = 0.1 
        self.AddEquity("QQQ", Resolution.Minute)
        self.symbol = Symbol.CreateOption("QQQ", Market.USA, OptionStyle.American, OptionRight.Call, 174, datetime(2020, 4, 17))
        self.AddOptionContract(self.symbol, Resolution.Minute) 

    def OnData(self, data: Slice):
        if not self.Portfolio.Invested:
            if self.symbol in data.QuoteBars:
                target = PortfolioTarget.Percent(self, self.symbol, self.targetPercent) 
                self.MarketOrder(self.symbol, target.Quantity)

    def OnOrderEvent(self, orderEvent):
        if orderEvent.Status == OrderStatus.Filled:
            portfolioWeight = self.Portfolio[orderEvent.Symbol].HoldingsCost / self.Portfolio.TotalPortfolioValue 
            self.Debug(f"\nActual portfolio weight of {orderEvent.Symbol.Value} after order fill: {portfolioWeight:.2%}. Target percent was {self.targetPercent:.2%}.\n")

System Information

QC Cloud

Checklist

[X] I have completely filled out this template
[X] I have confirmed that this issue exists on the current master branch
[X] I have confirmed that this is not a duplicate issue by searching issues
[X] I have provided detailed steps to reproduce the issue

ArthurAsenheimer commented 2 years ago

I thought about it and came to the conclusion that it might be more robust to use security.FillModel instead since users can have their own custom fill models where the fill price could differ from the ask/bid price. Either way, the portfolio percentage should always be less or equal to the target percentage. What do you think @Martin-Molinero ?

Martin-Molinero commented 2 years ago

Hey @ArthurAsenheimer ! Tricky, yes, I believe your right the different your seeing seems to be from using Security.Price when determining the quantity and filling the position with a market order against the Security.AskPrice

the portfolio percentage should always be less or equal to the target percentage

Right, it's an interesting case. I believe this is why we added the FreePortfolioValue/FreePortfolioValuePercentage https://github.com/QuantConnect/Lean/blob/master/Common/Interfaces/IAlgorithmSettings.cs#L62

Instead we should use security.AskPrice if target.Quantity > security.Holdings.Quantity (i.e. delta > 0) and otherwise security.BidPrice;

For this particular trade the spread was very big causing the jump against the target. Currently could override the securities buying power model to change the behavior, but maybe there is room for a smarter PortfolioTarget.Percent calculation, or at least allowing some further finer grain configuration. Not sure it should just use the Ask/Bid prices because you can argue it might cause a similar issue with this big spread cases (what if the algo is trading limit orders, etc)

AlexCatarino commented 2 years ago

If the algorithm calculated the order quantity with the bid/ask, it could not solve the problem, because the spread can be small (bid and/or ask price close to the trade price) when the quantity is calculated and increase when the order is filled. I think spread can vary rapidly with illiquid assets. @LouisSzeto has implemented the SpreadExecutionModel to address this problem from the execution perspective.

ArthurAsenheimer commented 2 years ago

Hey @Martin-Molinero , @AlexCatarino , Thanks a lot for your feedback. I see now there is no simple solution for this problem.

If the algorithm calculated the order quantity with the bid/ask, it could not solve the problem, because the spread can be small (bid and/or ask price close to the trade price) when the quantity is calculated and increase when the order is filled.

That can be solved by making sure the PortfolioTarget is always up to date and was calculated on the basis of the last known price.

I think spread can vary rapidly with illiquid assets. @LouisSzeto has implemented the SpreadExecutionModel to address this problem from the execution perspective.

My concern is not so much the large bid-ask spread, but rather the effect that the resulted percentage is significantly larger than the target percentage. I don't expect a 100% match here. That's actually not possible in most cases. But in case of doubt, the result should be always $\le$ target percent. I'm okay if the algorithm buys only 9% instead of 10% but not the other way around.

I believe this is why we added the FreePortfolioValue/FreePortfolioValuePercentage

That would avoid some of the issues with insufficient buying power, but it would be a constant percentage/dollar amount and we don't know in advance how much we need. The difference depends on the target percentage. In the example of my original post the target percent was just 10% and the PortfolioTarget.Percentgave us 14%. If we want to invest 100% in that QQQ option contract, PortfolioTarget.Percent would give us 238% (!). So FreePortfolioValuePercentage cannot really handle this. In addition, the FreePortfolioValuesettings would be applied to all securities which may not be intended.

Not sure it should just use the Ask/Bid prices because you can argue it might cause a similar issue with this big spread cases (what if the algo is trading limit orders, etc)

I think you brought up a good point with limit orders. The Ask/Bid may not always be the best choice here. But I don't see a good reason why the Close is supposed to be more suitable. Submitting limit orders requires to pass a price either way, so I think this issue is more relevant for market orders. But that brings me to the idea that we could add something like referencePrice as an optional argument to PortfolioTarget.Percent so that it would perform the calculations based on referencePrice and if no value is given it would use the default calculation. But even for the default calculation, I still think that the Bid/Ask give us a more robust and reliable result.

Ultimately, it comes down to solve the following optimization problem (here formulated for positive targets);

$$ \begin{align} &\max && q \ &\text{s.t.} &&p(q) \le p* \end{align} $$

where $p$ is the target percentage, $q$ the quantity and $p(q)$ refers to the resulted portfolio percentage, i.e. $p(q)$ = `q estimatedFillPrice / Portfolio.TotalPortfolioValue`. Put simply, we want to maximize the quantity subject to the given condition after which the portfolio percentage should be less or equal the target percentage. Essential here is the constraint " $\le$ target percentage". Every quantity that does not respect this constraint should be considered as 'out of scope' in my opinion.

Now everything depends on what we choose for estimatedFillPrice and since we don't know the fill price in advance, we should use something that is as close as possible to the expected/actual fill price which is usually the Bid/Ask. But there might be more generic solutions like extending Security.FillModel by a property that returns the (theoretical) fill price and use that for PortfolioTarget.Percent.

Martin-Molinero commented 1 year ago

TODO: review and adjust PortfolioTarget behavior to use ask/bid/trade when appropriate & available (as FillModel does) using a market order assumption. Believe this adjustment will end up being in the buying power model margin requirement calculation

QuantConnect / Lean