Add ability to have pending funds in the CashBook

[mchandschuh]

It is typical of US equity accounts to require 3 days after a sale before the funds 'clear' and are usable in your account. They still count towards your portfolio value, but you just can't use them to actually purchase any more securities.

What's missing from the engine in order to support this is a concept of pending margin in the Portfolio. Fills would still happen as usual and the cash would be applied directly the the cash book, but in the MarginRemaining calculation we would subtract out the pending margin items.

Maybe something like:

// in SecurityPortfolioManager
List<PendingMargin> PendingMargin

public void ScanPendingMargin(DateTime currentTimeUtc)
{
    // loop over pending margin items and see if we've passed the time
    // if we've passed the time remove it from the pending margin list
}

// SecurityPortfolioManager.MarginRemaining would subtract the pending margin items

public class PendingMargin
{
    public DateTime ApplicationTimeUtc;
    public decimal Value;
}

// in AlgorithmManager time loop we need some way to 'poke' the PendingCash
// in the section where we perform updates to security prices
algorithm.Portfolio.ScanPendingMargin(timeSlice.Time);

[dpavlenkov]

My initial thought on this is: we don't have pending margins, but we do have pending settlements which are applied to cashbook. So cash in cashbook needs to be divided into settled cash and pending cash. Then you can calculate your available margin from settled cash. Am I on the right track?

[jaredbroad]

We use cash to calculate portfolio value, so if the "cash" value changed because some of it was pending -- it would make the equity chart jump downwards.

In the brokerages they simply display it as part of your total cash balance -- separating "Cash" and "Buying Power" which for us is effectively margin. So simply modelling it as a temporary dip in buying power/margin is simplest IMHO.

[mchandschuh]

Agreed, we use cash as your cash balance and margin as buying power. So, technically, the cash is there, you just can't use it to buy more stuff (margin). But actually, @Nik1Milev correctly pointed out on the QC forums that you can use the cash, you just can't sell things you buy with that cash until it's been settled, tricky.

[jaredbroad]

Alternatively we could do settled cash modelling, which is more accurate -- we'd just need to reflect it in the Portfolio Value calculations (everywhere cash is used would be replaced with Cash + UnsettledCash)

From Tradier: How does Regulation T govern the usage of funds in a cash account?

When shares of stock or option contracts are sold in a cash account, there is a settlement period during which time the cash and the stock or options are exchanged. For stocks, this settlement period is 3 business days. For options it is 1 business day. When a stock or option is sold, these unsettled funds are available to use for purchase. However, if the newly purchased stock or option is sold prior to the settlement of the previous sale, this transaction violates Regulation T. Violations of Regulation T could result in restrictions being placed on the account. Please click here to see the Regulation T section on the SEC Website (http://www.sec.gov/investor/alerts/cashaccounts.pdf).

https://brokerage.tradier.com/support

[dpavlenkov]

Additionally, only cash accounts are restricted to trading on settled cash. Margin accounts are allowed to trade with unsettled funds and are allowed 3 good faith violations per year. Pattern day traders are allowed to trade with unsettled funds, moreover, their initial margin is only 25% (overnight positions are subject to 50% maintenance). So, calculating available margin from settled cash applies to cash accounts and to margin accounts that want to play it safe. Pattern day traders (I think we want to account for those) need more involved rules.

In view of this, I think we do need Settled and Unsettled cash, and we can cover the simple case with Pending settlements. We can also initialize SecurityMarginModel with dedicated intialMargin.

These rules apply to US accounts. Other jurisdictions will have different rules, and they'll need to be covered by more models like IAccountMarginModel or something.

[jaredbroad]

Also from Tradier: the funds are released on the third morning after the sale, so a Monday 2pm sale would be released Thursday morning by 8am - (i.e. its not 72 hours, but third morning)

[mchandschuh]

@dpavlenkov - Instead of an account level margin model, I was thinking of keeping it security based, since the regulations change based on country/security type. So for equities, its the 3 day waiting, but for options it's available the next day (I believe). I'm thinking an extension of IMarginModel that maybe applies the funds to the cash book or to unsettled funds depending on the model. The Portfolio object could then manage the unsettled funds (assuming they state when they should be settled)

I'm thinking of some brokerages that allow international trading, such as IB. I could be trading US equities, Australian equity options, and Forex futures all from the same account.

[dpavlenkov]

No worries, I didn't intend to scrap security margin model. I suppose we could initialize security margin models differently depending on user requirements.

[jaredbroad]

I agree it should be security based -- implemented on the MarginModel -- but also default to using a MarginModel which detects cash balance under $25k + the account type to determine PDT restrictions.**

Then we'd need an QCAlgorithm method SetAccountType() which takes the account type -- cash / margin -- and the BrokerageSetupHandlers would need to set it from IBrokerage when starting up the algorithm.

I know with Tradier we can get this information as BrokerageData. Not sure how we'd get it with IB yet.

\ Current Lean design is "default to US but allow extension to other countries"

[dpavlenkov]

SetAccountType will make things easy to setup for the user. Internally, we have two concerns: 1) initial and maintenance margin requirements 2) settled and unsettled cash. To handle this flexibly, let's have an interface code named "Available Margin Calculator" which will decide whether to use unsettled funds for trading depending on security type or even per security. Then the function of SetAccountType will be to initialize algorithm with appropriate calculator, with additional ability initialize algorithm with user-defined calculators.

[jaredbroad]

@dpavlenkov -- the margin calculation work (initial vs maintenance) has already been implemented. The "Available Margin Calculator" is "GetMaintenanceMargin" and its all done in the MarginModel

I dont think we need any new interfaces, just better modelling on the default MarginModel. This part of code is pretty complex. I'd recommend digging into it before you start.

[dpavlenkov]

I'm talking about decision whether to use unsettled funds for trading. Do you propose that margin models make that decision?

[jaredbroad]

Ah good point -- to start lets model the first half -- "blocking unsettled funds from trading" (which is a big task itself given brokerage concerns). Then once its done we can solve how to trade with unsettled funds.

(IMHO git issues should be as light and minimum scope as possible - so they can be built and shipped with low risk with 2-3 hours work maximum)

[dpavlenkov]

Yes, no problem, it's just thinking of bigger scope helps direct the small scope. It started with an idea, now we have a plan. So, if I understood this correctly, for the first iteration:

1) Separate cash into Settled and Unsettled 2) Implement pending settlements and have them applied to cashbook, which moves unsettled cash to settled 3) Margin calculation methods that now look at total cash need to restrict to settled cash

[jaredbroad]

1) CashBook (as is now) and the a new UnsettledCashBook instance (both using CashBook type, living on Portfolio) 1.2 Pass cash change request into a new method: IMarginModel.ApplyFunds( SecurityPortfolioManager, Order, decimal funds) anywhere the Cash property is directly set, and apply the Security's Margin filter on where the cash goes. E.g. FOREX cash is immediately applied.

4) Update total portfolio value calcs to include unsettled cash so equity is smooth. 5) Update API to have SetAccountType (AccountType.Margin, AccountType.Cash) 6) Update BrokerageSetupHandler & implementations (tradier & IB) to set account type in the algorithm based on logins!

[dpavlenkov]

Well, like you said, it's better to keep issues small, so perhaps make 5 and 6 another issue?

[Nik1Milev]

Hi All,

I really enjoyed reading the email trail - the moment a question pops up in my head, it is answered on the next email.

I would like to share my 2 cents: It seems to me it is better to leave a choice to developers:

• mode 1, no changes - use it as it is now, so cash is available immediately and no regulation T is considered
• mode 2, the one you guys discussed.

Mode 1 is important to me, because as a developer i want to know the theoretical performance of the algorithm. E.g.: I would assume that it has a draw-down 20%, this is what i can expect when i enter the market, since i might be entering just before the draw-down.

The whole point of mode 2 is to make it more realistic and if one wants to implement money management, he needs to run algos in this mode.

Another addition in order to make it more realistic: when we run algos we assume that we can buy all the shares we want, which sometimes is many times more than the actual shares traded for entire day. So, this is good for theoretical model, but for real run (and live runs) I would suggest to allow setting a ceiling/threshold for max shares to buy/sell as % of daily traded. E.g. If we buy shares 2-3%of total volume, we perhaps will not affect the price and it is reasonably true. But if algo buys 80% of all available shares traded that day, than the price will definitely be very different.

Thanks and good luck

Nik

On Thu, Jul 30, 2015 at 4:51 PM, Jared notifications@github.com wrote:

1) CashBook (as is now) and the a new UnsettledCashBook instance (both using CashBook type, living on Portfolio) 1.2 Create IMarginModel.ApplyFunds( SecurityPortfolioManager, Order, decimal funds) anywhere the Cash property is directly set, and apply the Security's Margin filter on where the cash goes. E.g. FOREX cash is immediately applied.

4) Update total portfolio value calcs to include unsettled cash so equity is smooth. 5) Update API to have SetAccountType (AccountType.Margin, AccountType.Cash) 6) Update BrokerageSetupHandler & implementations (tradier & IB) to set account type in the algorithm based on logins!

— Reply to this email directly or view it on GitHub https://github.com/QuantConnect/Lean/issues/146#issuecomment-126503061.

[AnObfuscator]

Ah good point -- to start lets model the first half -- "blocking unsettled funds from trading" (which is a big task itself given brokerage concerns). Then once its done we can solve how to trade with unsettled funds.

I think this is the right call. The thing is, unsettled funds can be used immediately to purchase new stocks, but those stocks then can't be sold until the funds are settled. For day trading, this is terrible; for buy-and-hold, maybe not so much.

There's a lot of variations that could be considered. Getting a solid model for unsettled funds is the first and most important step. :)

[mchandschuh]

@Nik1Milev - we currently have a plugin point to define your own fill models, via the ISecurityTransactionModel interface. In there you could define your fill behavior to be dependent on the recent volume in the security, even have low volume negatively impact the fill price.

@dpavlenkov - We can start with the core and add the API goodies on top later.

@AnObfuscator - Always words of wisdow :) I agree, let's get a solid foundation in with the correct abstraction and making it do whatever it needs to do later should be simple.

[jaredbroad]

@dpavlenkov -- my logic for doing 4,5,6 is to "completely" finish the first part of it. Anything we do now we have to implement for backtesting and live :) Its twice the work but it means the code can be used in live trading as well.

[jaredbroad]

After thought -- since this can be set manually by the user with SetAccount type we can skip the parts where its automatically set by the brokerage (step 6). That can be a "nice to have" feature for next iteration.

[dpavlenkov]

Considering that this affects live trading most of all, we need to make sure we can sync the balances we need. Tradier can give us available funds, total funds, unsettled funds. Interactive brokers through krs.ats.ibnet can give us total funds and available funds. krs.ats.ibnet does not appear to implement reqAccountSummary, which would give us more info, and probably what we need. I don't see the point of doing this for backtesting without having a live model to fit to.

EDIT: reqAccountSummary is only for financial advisor accounts, so we have to do with account updates only.

[jaredbroad]

Good point! I forgot that there may be unsettled balances on deploying the algorithm.

[mchandschuh]

I believe the IB implementation would send it back via account properties, this is how we currently get the cash. Here's an incomplete list of the available account properties.

[dpavlenkov]

https://www.interactivebrokers.com/download/newMark/PDFs/APIprintable.pdf page 100 has good descriptions of account properties.

same thing in html

[dpavlenkov]

I checked the properties on my paper and live accounts and I don't see how we can get unsettled cash directly. Indirect way is to scan executions and guess.

[dpavlenkov]

We have three brokerages, and they provide unsettled cash differently. Backtesting brokerages models it from fills. Tradier brokerage provides it directly. IB provides executions and we can model it from that. So I think we need to let brokerages manage unsettled cash and access it through IBrokerage.GetUnsettledCashBalance() . To model unsettled cash internally, a brokerage could populate and maintain a list of PendingSettlements and drop each one at expiration time.

[mchandschuh]

In 1.2, we should move the ApplyFunds method to a new ISettlementModel which would live on the Security object.

[jaredbroad]

Closed in #ca42829