LIFO/"breakeven" pps for `ib`

goodboy commented 2 years ago

Proper attempt at #307 and a local "trades" ledger system using .toml files under <confdir>/ledgers/ where files will be named something like: trades_<brokername>_<accountname>.toml

So far ledger contents looking something like this:

Click to expand!

```toml [ib.20220512.977661167] accountId = "DUDUDUDUD" acctAlias = "" model = "" currency = "USD" fxRateToBase = 1.3046 assetCategory = "FUT" symbol = "MGCM2" description = "MGC 28JUN22" conid = 430360671 securityID = "" securityIDType = "" cusip = "" isin = "" listingExchange = "NYMEX" underlyingConid = 79702479 underlyingSymbol = "MGC" underlyingSecurityID = "" underlyingListingExchange = "" issuer = "" multiplier = 10 strike = "" expiry = 20220628 tradeID = 977661167 putCall = "" reportDate = 20220512 principalAdjustFactor = "" dateTime = "20220512;115003" tradeDate = 20220512 settleDateTarget = 20220513 transactionType = "ExchTrade" exchange = "NYMEX" quantity = 1 tradePrice = 1834.3 tradeMoney = 18343 proceeds = -18343 taxes = 0 ibCommission = -0.77 ibCommissionCurrency = "USD" netCash = -97.77 closePrice = 1824.6 openCloseIndicator = "O" notes = "" cost = 18343.77 fifoPnlRealized = 0 fxPnl = 0 mtmPnl = -97 origTradePrice = 0 origTradeDate = "" origTradeID = "" origOrderID = 0 clearingFirmID = "" transactionID = 3896422621 buySell = "BUY" ibOrderID = 398090454 ibExecID = "0000e1a7.627cece5.01.01" brokerageOrderID = "00dd230498015d2347c93b9.0001" orderReference = "" volatilityOrderLink = "" exchOrderId = "00d345980012a0234dd49[2]" extExecID = "237147708B" orderTime = "20220512;115003" openDateTime = "" holdingPeriodDateTime = "" whenRealized = "" whenReopened = "" levelOfDetail = "EXECUTION" changeInPrice = 0 changeInQuantity = 0 orderType = "LMT" traderID = "" isAPIOrder = "Y" accruedInt = 0 serialNumber = "" deliveryType = "" commodityType = "" fineness = 0.0 weight = "0.0 ()" ```

And, as of right now pps.toml entries lookin more or less like this (example only):

[ib.algopaper."mnq.globex.20220916"]
size = 58.0
be_price = 11703.458965517251
bsuid = 497954580
expiry = "2022-09-16T00:00:00+00:00"
clears = [
 { tid = "0000e1a7.629f3c2d.01.01", cost = 0.57 },
 { tid = "0000e1a7.629fa5c0.01.01", cost = 0.57 },
 { tid = "0000e1a7.62a0a898.01.01", cost = 0.57 },
 { tid = "0000e1a7.62a0fb24.01.01", cost = 0.57 },
]

REQUIREMENT:

[x] this puts us on 3.10+ due to use of the new structural pattern matching syntax so we should probably do a PR to enforce that before merging this.. (landed in #344)

TODO: (oustandings moved to #345)

[x] pps.toml local position tracking/summary file
- [ ] manual edits by the user and just have the ledger parser(s) complain on mismatches
- [x] better format in terms of toml data structures so that it's easier to read the pp set by eye:
- [x] include each transaction as an array of tables which include id, cost, (date-)timestamp, maybe something else?
- see below, think this is mostly there as a POC
- [x] make fields under each pp for a breakeven_price
- [ ] include a lifo_price?
- [ ] make ^ prices all arrays which increment with each new transaction/fill entry
[x] pp parsing using LIFO style and since last net zero style reporting
[x] appends from session trades capture via Client.trades() method
[x] integration with PostionTracker UX on charts

Probably as follow up PR(s): -> See #345 for all deferrals and detailed write up of all these

[ ] extend this to the kraken ledger parsing and generalize into a top level module?
[ ] piker ledger subcommand?
[ ] piker pps subcommand with opens pps.toml in editor/pager?
[ ] roll our own toml encoder?
- after much pain and anguish getting the toml encoder to output things sanely, i think it's just worth writing a super fast custom encoder that writes adhoc-ly in the format we want for pps.toml, the reader-writer separation is already what tomli does (the fastest pure py reader soon to land in the stdlib) and and then we can just drop toml outright and worry about getting a general encoder later.
- support better indentation per broker-account for better readability
[ ] as per bullets in TODO above, we should support pp liftetime "lines" more or less as super simple FSP style graphics that show the beginning, changes, and end to a given pp such that users can easily see past trades and their state changes on top of historical data
[ ] add proper paper engine support so that user's can trade their forward testing systems like bosses
[ ] docs / recommendations for storing all ledger / pps.toml updates in git

goodboy commented 2 years ago

Marking this ready to get some eyes, but it's still super ib-specific and I want to still finesse a buncha stuff and make follow up issues for a more general solution / API for ledgers.

goodboy commented 2 years ago

Aight, as of right now pps.toml entries lookin more or less like this (example only):

[ib.algopaper."mnq.globex.20220916"]
size = 58.0
be_price = 11703.458965517251
bsuid = 497954580
expiry = "2022-09-16T00:00:00+00:00"
clears = [
 { tid = "0000e1a7.629f3c2d.01.01", cost = 0.57 },
 { tid = "0000e1a7.629fa5c0.01.01", cost = 0.57 },
 { tid = "0000e1a7.62a0a898.01.01", cost = 0.57 },
 { tid = "0000e1a7.62a0fb24.01.01", cost = 0.57 },
]

includes a table of .clears which is the minimal amount of data parsed from the corresponding backend ledger in order to produce the high level details.

As further refinemen we could probably,

[x] add a datetime field the clears table?
[ ] (will defer) include an array of price histories?
[x] show size/price values for each clear?

pikers / piker

LIFO/"breakeven" pps for `ib` #336