Track & show deposited lots

simonmichael commented 5 years ago

Part of #1015. This issue is about adding basic awareness of lots to hledger. A lot is some quantity of a commodity transacted at a certain price on a certain date. Tracking lots is a prerequisite for (automated) reporting of realised capital gains. This issue's scope is intentionally limited to "deposits", ie adding lots to an account, and reporting that they are there. See also #1023.

Bounties for this issue: $65 @pkel

Questions

What's an example of the opposite of a "lot deposit" ?

Eg, some kind of liability, or option, where you owe a certain kind of lot.

Is any new journal syntax needed for this issue ?

Semantically/at user level, what generates a lot deposit rather than an ordinary amount deposit ?

In Ledger

Use of @ or @@ or {} or {{}} instead of @= or @@= or {=} or {{=}} ? The first four are considered variable, the last four are fixed. --Fixing Lot Prices.

This test journal seems to show that all of them generate lot prices; only @ generates lot dates; only {} generates capital gains. As usual, I am completely confused.

"Fixated pricing ... still plays a role in this scheme" --Complete control over commodity pricing.

"When a transaction occurs that exchanges one commodity for another, Ledger records that commodity price not only within its internal price database, but also attached to the commodity itself. Usually this fact remains invisible to the user, unless you turn on --lot-prices to show these hidden price figures." --Commodity prices.

Cost (@) and "lot price" ({}) are mentioned as different concepts, not just alternate spellings, but "Functionally, however, there is no difference" --Prices versus costs. "price annotations and costs are largely interchangeable and a matter of preference" --Fixated prices and costs.

"You can also associate arbitrary notes for your own record keeping in parentheses, and reveal them with --lot-notes." --Lot notes. Syntax: AMOUNT {LOTPRICE} [LOTDATE] (LOTNOTE).

In Beancount

Beancount 2: https://beancount.github.io/docs/how_inventories_work.html Uses @ and {...}. It's still complicated, see https://www.reddit.com/r/plaintextaccounting/comments/ltqirt/one_lot_per_tx_or_how_to_handle_cost_basis_for

Beancount 3 (proposed): http://furius.ca/beancount/doc/v3-booking (uses @ with a bit of {...} mainly for backward compatibility)

In beans

Transaction prices are not declared explicitly, but inferred for every posting not already in the target commodity, from the market prices at posting date. With these, unrealized gains are calculated.

There's no way to select lots being sold, and realized gains are not calculated (?).

In lotter

It reads a journal of just the basic transactions and costs, and writes new journal entries enriched with lot info (?)

"lotter has a good approach because it separates the "what happened" from "what does it mean?"."

In hledger

Presently:

hledger doesn't have a special concept for lots, you model them manually with subaccounts. The close command and a powerful text editor are useful for constructing journal entries.

In future:

Any multi-commodity transaction (with explicit or inferred transaction price) ?

An explicit transaction price is written (@ or @@) ?

A transaction-priced increase to assets transferred from anywhere (eg revenue) ?

A transaction-priced increase to assets transferred from another asset ?

Some special syntax to activate lot tracking ?

Internally, when to use lots rather than ordinary amounts ?

Lots for everything ?

Lots for postings with a transaction price ?

Lots for postings of a commodity which has a known market price on the posting date ?

How to represent lots internally ?

Eg:

-- | An amount of some commodity transacted at a certain price on a certain date. 
data Lot = {
   lamount   :: Amount
  ,lprice    :: Price
  ,ldate     :: Day     -- ^ same as posting date
  ,lsequence :: Int     -- ^ sequence number ? (within lots of this commodity at this price on this day)
  }

-- | Lots in date and intra-day order, separated by commodity.
type Lots = Map CommoditySymbol [Lot]

data Account = Account {
   aname                     :: AccountName    -- ^ this account's full name
  ...
  ,aebalance                 :: MixedAmount    -- ^ this account's balance, excluding subaccounts
  ,aibalance                 :: MixedAmount    -- ^ this account's balance, including subaccounts
  ,alots                     :: Lots           -- ^ any priced lots held in this account
  }

Would alots equal aebalance, or augment it, or replace it ?

But actually every Amount already contains a (possibly empty) Price, used to represent transaction price. We could add the date (and sequence number?) there. Then Amount could represent an ordinary amount or a lot, and MixedAmount could contain a mixture of amounts and lots.

Is storing date in Amount redundant with the Posting's date ?

We think no, these could be different - see the stock split example below.

When a posting has both a transaction price and a market price, which one is used as lot price ? Which one is used to calculate capital gains ?

I think the transaction price. Ie the price you paid, not the prevailing market price on the day.

What are some basic ways of showing lots ?

1018 Ledger-style balance --lots, register --lots reports
print ?
hledger close --close --show-costs ACCTS

Is revaluation due to market price changes, different from unrealized capital gains ?

I think not. More discussion on #1029.

simonmichael commented 2 years ago

(I probably got something wrong, but hopefully this gives the idea.)

pkel commented 2 years ago

I wrote a comment on a different issue about a year ago which might be relevant here. https://github.com/simonmichael/hledger/issues/1029#issuecomment-776220754

It's yet another proposal for dealing with lots. In my* proposal, hledger creates and consumes lots implicitly according to an accounting rule (FIFO, LIFO, ...). It requires neither curly braces nor @-syntax. Lots live in a trading account on the equity side. This enables to move assets, e.g. from one bank to another, without touching the lots. Balance of the trading account after price conversion resembles unrealized capital gains. I think --infer-equity is a partial implementation of my proposal.

*: I probably got the idea from someone else but forgot to take note.

zhzy0077 commented 2 years ago

Some of my scenarios - just for reference:

I do balance assertions at a weekly basis, to find out if the broker reinvests dividends. However balance assertions cannot work on a aggregated manner, i.e. following sample won't work

2020-11-02 * Buy
  Assets:Investments:BTC:20201102   +1 BTC @ 100 USD
  Assets:Cash
2020-11-03 * Buy
  Assets:Investments:BTC:20201103   +1 BTC @ 200 USD
  Assets:Cash

2020-12-01 * Assertion
  Assets:Investments:BTC                   = 2 BTC

Proposal: allow to assert on subtotal of an account.

I enabled --strict of hledger check command, so I have to declare all dates(or lots) as an account or I will see undeclared account error.

Proposal: add a flag to set no validation for a specific account.

simonmichael commented 2 years ago

Thanks for this input.

The first part is already possible: use =* (https://hledger.org/1.25/hledger.html#assertions-and-subaccounts https://hledger.org/1.25/hledger.html#assertions-and-subaccounts).

lestephane commented 2 years ago

The tracking of lots using only the cost basis and sale price infomation will indeed make things easier.

Would balance assignments then be a thinkable way of handling stock splits?

Consider the recent Amazon split, the only information I have in the Revolut csv is

Date,Ticker,Type,Quantity,Price per share,Total Amount,Currency,FX Rate
2022-06-06T05:20:19.099196Z,AMZN,STOCK SPLIT,9.37032785,,$0,USD,1.08

The quantity 9.37... indicates how many shares are present in the investment account after the split happened. There is no other information. If the symbol is know to affect a tracked lot, then one could imagine the following to communicate the new count of shares.

2022-06-06  AMZN Stock Split
  Assets:Investments:AMZN  = 9.37032785 AMZN

Arguably, the csv from Revolut is pretty bad, offering no information as to what the split ratio was, but I wanted to get the conversation going on splits, which will definitely influence the handling of lots. Who knows, maybe even a special balance assignment syntax will be needed... But I dare not think of syntax changes anymore, as it makes the harmonization between command line accounting tools exponentially more difficult

alensiljak commented 1 year ago

I'd add an article on Tax lots, which seems the best description of the Lot implementation in PTA apps:

https://www.investopedia.com/terms/t/taxlotaccounting.asp

simonmichael commented 1 year ago

Excellent link, thanks.

simonmichael / hledger