simonmichael / hledger

Robust, fast, intuitive plain text accounting tool with CLI, TUI and web interfaces.
https://hledger.org
GNU General Public License v3.0
2.91k stars 315 forks source link

doesn't fully support Ledger's {} price syntax #1084

Open mapreri opened 5 years ago

mapreri commented 5 years ago
% cat foo.ledger
2019-04-22 Foo
      ;Program: Bar
      ;Entity: Baz
    Accrued:Accounts Payable:Bar      £-154.83 {=$1.29847} @ $1.29847
    Expenses:Bar:A                    £154.83 {=$1.29847} @ $1.29847
% ledger --file foo.ledger balance
            £-154.83  Accrued:Accounts Payable:Bar
             £154.83  Expenses:Bar:A
--------------------
                   0
% hledger-ui -f foo.ledger                      
hledger-ui: /home/mattia/devel/reproducible/ledger/foo.ledger:4:60:
  |
4 |     Accrued:Accounts Payable:Bar      £-154.83 {=$1.29847} @ $1.29847
  |                                                            ^
unexpected '@'
expecting ';', end of input, or newline
% hledger-ui --version
hledger-ui 1.12.1

Afaik that syntax is common when dealing with invoices in foreign currencies that you want to keep track of without converting them in your ledger entry.

However, it seems hledger doesn't cope with that at all.

simonmichael commented 5 years ago

Hi @mapreri.. the {} syntax is used by Ledger, but not hledger. https://hledger.org/journal.html#transaction-prices describes the syntax we currently support, and this seems related at https://hledger.org/faq.html#functional-differences : "Ledger allows amounts to have a fixed lot price (the {} syntax ?) and a regular price in any order (and uses whichever appears first). hledger requires the fixed lot price to come last (and ignores it)."

simonmichael commented 5 years ago

So, I think you can work around by just putting the {} last:

2019-04-22 Foo
      ;Program: Bar
      ;Entity: Baz
    Accrued:Accounts Payable:Bar      £-154.83 @ $1.29847 {=$1.29847}
    Expenses:Bar:A                    £154.83 @ $1.29847 {=$1.29847}
mapreri commented 5 years ago

So, I think you can work around by just putting the {} last:

Mh, that's indeed the case, swapping the two syntaxes does appear work, thank you for the hint!

I'll let you choose the fate of this bug (left opened as a feature request to also support the {} syntax, or close it).

simonmichael commented 5 years ago

It's a good question. hledger so far ignores {} because I don't like that syntax - I find it overcomplicated and unnecessary. IIRC it does exactly what hledger's @ does already. (I forget exactly how Ledger's @ is different from ours.) More Ledger compatibility is good to have, but more confusion and complexity in hledger is not. Investigation, opinions welcome.

ony commented 4 years ago

As far as I remember {} produce additional postings for capital gains and capital loss. I simulate that with next transactions

2018-03-31 Report
    Income:Stocks  $136.57  ; Unrealized capital loss
    Assets:Stocks  -1 AMZN @ $1,583.91  ; corresponds to {=$1,583.91} - original price
    Assets:Stocks  1 AMZN @ $1,447.34

Or

2018-03-31  Sell
    Income:Stocks  $136.57  ; Realized capital loss
    Assets:Stocks  -1 AMZN @ $1,583.91  ; original price
    Assets:Bank:Current  $1,437.34  ; $1,447.34 - $10
    Expenses:Fee  $10

In second case some transaction might not explicitly mention new price. Thus sometimes I kinda merge them and get ugly:

2018-03-31  Sell
    Income:Stocks  $136.57  ; Realized capital loss
    Assets:Stocks  -1 AMZN @ $1,583.91
    Assets:Stocks  1 AMZN @ $1,447.34
    Assets:Stocks  -1 AMZN @ $1,447.34
    Assets:Bank:Current  $1,437.34
    Expenses:Fee  $10

Effectively when you write

    A  -1 AMZN {=$1,447.34} @ $1,583.91

Is roughly equivalent of

    I  $136.57  ; ($1,583.91 - $1,447.34) * 1
    A  -1 AMZN @ $1,583.91
    ; transaction balance = $1,447.34
simonmichael commented 4 years ago

@ony: thanks, very helpful examples. I will remember: "Ledger's {} is like @ except it also generates capital gain/loss transactions".

Do your manual entries get very tedious, eg if doing a lot of stock trading ?

We may end up having to support {}. But I have always found it really confusing. Is that true, or just me, or just lack of the right docs ?

Can anyone think of a more intuitive syntax for this, if you were designing from scratch ?

simonmichael commented 4 years ago

PS and what's the difference between {AMT} and {=AMT} ?

simonmichael commented 4 years ago

If I'm reading https://www.ledger-cli.org/3.0/doc/ledger3.html#Fixing-Lot-Prices right:

See:

This transaction actually introduces a new commodity, ‘GAL {=$2.29}’, whose market value disregards any future changes in the price of gasoline.

If you do not want price fixing, you can specify this same transaction in one of two ways, both equivalent (note the lack of the equal sign compared to the transaction above):

2009/01/01 Shell
    Expenses:Gasoline             11 GAL {$2.299}
    Assets:Checking

2009/01/01 Shell
    Expenses:Gasoline             11 GAL @ $2.299
    Assets:Checking

That reads backwards to me. Going by Ledger's behaviour, I think of the one without the = as fixed for all time, and the one with = as affected by market price changes.

simonmichael commented 4 years ago

Related to #1029. I think we can consider this a sub-task of that.

simonmichael commented 4 years ago

But continuing this train of thought:

As I understand it,

Ledger has ~four~six syntaxes for specifying price in a transaction: @, @@, { }, {= }, {{ }}, {{= }}. @@ is another sometimes more convenient form of @. { } is an alternate spelling of @ (I'm not sure why it exists).

So, semantically it has two kinds of transaction price: @ which will not generate capital gains transactions for that amount, and {= } which will.

hledger has one kind of transaction price. I don't yet see that two are needed. I think that market price (P) records are the thing that should cause capital gains transactions.

Ledger's manual says only:

There is no difference in meaning between these two forms. Why do both exist, you ask? To support things like this:

2009/01/01 Shell
    Expenses:Gasoline             11 GAL {=$2.299} @ $2.30

I think this is not a real example, or can someone explain it ? Or describe other situations where two separate prices on a posting are desirable ? Would stock options be one such ?

simonmichael commented 4 years ago

This is seems interesting and possibly related, from https://www.cointracker.io/blog/new-irs-cryptocurrency-tax-guidance#specific-identification . I've added our terminology in brackets:

To successfully identify a specific unit [lot], information must include:

  • the date and time each unit was acquired,
  • your basis [cost, transaction price] and the fair market value of each unit at the time it was acquired,
  • the date and time each unit was sold, exchanged, or otherwise disposed of, and
  • the fair market value of each unit when sold, exchanged, or disposed of, and the amount of money or the value of property received for each unit

Ie, two amounts are considered with each transaction: the amount exchanged, and the fair market value at that time. hledger currently doesn't expect you to record the market value along with the transaction, assuming that will be figured out from the market prices on that date.

adept commented 4 years ago

I think that IRS overcomplicates things here (and we should not). Let me try to annotate this piece of text the same way you did, while also considering this example:

2019-01-01 buy BTC
   cash  -$100
   crypto  1 BTC

2019-10-10 sell BTC
    crypto -1 BTC
    cash  $999

To successfully identify a specific unit [lot], information must include:

  • the date and time each unit was acquired [this is date of buy transaction, 2019-01-01],
  • your basis [price at the time of buying, $100] and the fair market value of each unit at the time it was acquired [IRS wants to know this presumably to ensure that you dont cook your books. We don't need this (even though price directive could be in the file and could provide this info)],
  • the date and time each unit was sold, exchanged, or otherwise disposed of [this is date of the sell 2019-10-10], and
  • the fair market value of each unit when sold, exchanged, or disposed of [again, this is IRS checking that you do not misreport the price to lower your taxes], and the amount of money or the value of property received for each unit [this is price at the price of selling, $999]

So I agree that amounts are considered, but I disagree with your description of them. I claim that they are the (qty*buy price) and (qty*sell price). Your capital gains or losses are (qty*sell price) - (qty*buy price), and this is what you pay capital gains tax on.

In the example above, your gains are $999*1 - $100*1 = $899

Specific identification is used not only for crypto, but for shares as well. Helpful example for shares that corroborates what I wrote: https://www.schwab.com/resource-center/insights/content/save-on-taxes-know-your-cost-basis (search for Specific identification method there)

So we need to either have a way to refer back to buy transaction or to include cost basis into the sell transaction. And ledger's syntax `{= cost basis}' could be useful here:

2019-01-01 buy BTC
   cash  -$100
   crypto  1 BTC

2019-10-10 sell BTC
    crypto -1 BTC @@ $999 {= $100}
    cash  $999

Or:

2019-01-01 buy BTC
   cash  -$100
   crypto  1 BTC

2019-10-10 sell BTC
    crypto -1 BTC {= $100}
    cash  $999

If this syntax is allowed, it should be possible to:

  1. automatically create Capital Gains transactions like Ledger does

  2. implement something like --lots from ledger or "account inventory" from beancount, where you compute lots (which are triples of (commodity, price, date)) on each buy and destroy some of them (or their parts) on each sale, and at the end report what is left.

adept commented 4 years ago

It is worth noting that a way to include cost basis in the sell transaction is most flexible -- you could, for example, record a sell (and capital gains) even if you have no record of buying the thing.

On the other hand, if you have all the buys recorded, then a way to record back to them would be more foolproof:

2019-01-01 buy BTC
   cash  -$1000
   crypto  10 BTC

2019-02-02 buy more BTC
   cash  -$2000
   crypto  10 BTC

2019-10-10 sell BTC
    crypto -10 BTC {2019-01-01}
    crypto -5 BTC {2019-02-02}
    cash  $15000

It would be more foolproof to use dates provided to refer back to buys to figure out that:

Cause if you record cost basis in the sell transaction directly, you can make a mistake like this:

2019-01-01 buy BTC
   cash  -$1000
   crypto  10 BTC

2019-02-02 buy more BTC
   cash  -$2000
   crypto  10 BTC

2019-10-10 sell BTC
    crypto -15 BTC {= $100}
    cash  $15000

And it would be sad if this would lead to capital gains of $13500 without any cross-checking, even though you never had 15 BTC bought at basis of $100, you had only 10 BTC.

simonmichael commented 4 years ago

So we need to either have a way to refer back to buy transaction, or to include cost basis into the sell transaction. And ledger's syntax `{= cost basis}' could be useful here.

I’ll respond to the rest later, but first this: wow, I had been considering use of {= } only in the buy transaction. Is it actually intended to be used in the sell transaction ? Only ?

rainbyte commented 4 years ago

I use the {} syntax to write transactions in this way:

2019-10-10 * Buy ABC
    crypto:abc       1 ABC @ 10 USD
    bank            -10 USD

2019-10-11 * Buy XYZ
    crypto:xyz          10 XYZ @ 0.95 USD ; 1 XYZ = 0.1 ABC
    expenses:loss     0.5 USD
    crypto:abc         -1 ABC {10 USD} [2019-10-10] @ 9.5 USD

Then command ledger -f crypto.journal bal 'crypto:xyz' --flat --lots would say:

10 XYZ {USD0.95} [2019-10-11]

I'm not using the {= 12.34 USD} syntax on any journal

adept commented 4 years ago

I’ll respond to the rest later, but first this: wow, I had been considering use of {= } only in the buy transaction. Is it actually intended to be used in the sell transaction ? Only ?

I mean, you can use it anywhere (i tried), but if you use it on sell transaction you get a useful side-effect of auto-generated capital gains transaction.

adept commented 4 years ago

I use the {} syntax to write transactions in this way: .... I'm not using the {= 12.34 USD} syntax on any journal I see that you are generating profit/loss postings manually, which kinda removes the need for {= ... }.

rainbyte commented 4 years ago

I didn't know that the {= ...} syntax could generate profit/loss postings, it is still a bit confusing for me.

From my pov the important part is being able to select an specific lot using {...} [...] @ ....

simonmichael commented 4 years ago

It’s just my interpretation of what ony said, don’t take my word for it. It is an extremely confusing UX and learning curve, our job is to clean this up if we can

rainbyte commented 4 years ago

I think that having multiple obscure ways to do the same thing is not so good.

Maybe manual profit/loss is not so difficult enough to justify a custom automatic syntax.

On the other hand, not having even a single way to select a lot could be a problem.


Maybe hledger can suggest writing profit/loss by hand upon finding the {= ...} syntax.

In that way the user learns the preferred way to represent that transaction.

dmage commented 2 years ago

I track my securities with ledger this way:

commodity CZK
    format 1,000.00 CZK

2019/11/20 * Buy
    Assets:Bank:Securities  479 "CZ5027" {1.0424 CZK} [2019/11/20]
    Expenses:Bank:Excess       0.69   CZK
    Assets:Checking         -500.00   CZK
    Expenses:Bank:Rounding     0.0004 CZK

2021/02/01 * Sell
    Assets:Bank:Securities  -479 "CZ5027" {1.0424 CZK} [2019/11/20] @ 1.1021 CZK
    Income:TY2021:Capital Gains  (-(1.1021 - 1.0424)*479 CZK)
    Assets:Checking          527.91   CZK
    Expenses:Bank:Rounding    -0.0041 CZK

Ledger ignores @ 1.1021 CZK, so it's a comment for me. Sometimes I would put Capital Gains 28.5963 CZK as I don't like to have expressions in my ledger file.

When I need to do taxes, I query income via

$ ledger -f foo.ldg bal TY2021 --unround
        -28.5963 CZK  Income:TY2021:Capital Gains

(--unround is handy for foreign currencies; for tax purposes they should be converted into the domestic currency at the annual exchange, so 0.005 EUR were about 0.13 CZK in 2021)