Track & show deposited lots

[simonmichael]

Part of #1015. This issue is about adding basic awareness of lots to hledger. A lot is some quantity of a commodity transacted at a certain price on a certain date. Tracking lots is a prerequisite for (automated) reporting of realised capital gains. This issue's scope is intentionally limited to "deposits", ie adding lots to an account, and reporting that they are there. See also #1023.

Bounties for this issue: $65 @pkel

Questions

What's an example of the opposite of a "lot deposit" ?

Eg, some kind of liability, or option, where you owe a certain kind of lot.

Is any new journal syntax needed for this issue ?

Semantically/at user level, what generates a lot deposit rather than an ordinary amount deposit ?

In Ledger

Use of @ or @@ or {} or {{}} instead of @= or @@= or {=} or {{=}} ? The first four are considered variable, the last four are fixed. --Fixing Lot Prices.

This test journal seems to show that all of them generate lot prices; only @ generates lot dates; only {} generates capital gains. As usual, I am completely confused.

"Fixated pricing ... still plays a role in this scheme" --Complete control over commodity pricing.

"When a transaction occurs that exchanges one commodity for another, Ledger records that commodity price not only within its internal price database, but also attached to the commodity itself. Usually this fact remains invisible to the user, unless you turn on --lot-prices to show these hidden price figures." --Commodity prices.

Cost (@) and "lot price" ({}) are mentioned as different concepts, not just alternate spellings, but "Functionally, however, there is no difference" --Prices versus costs. "price annotations and costs are largely interchangeable and a matter of preference" --Fixated prices and costs.

"You can also associate arbitrary notes for your own record keeping in parentheses, and reveal them with --lot-notes." --Lot notes. Syntax: AMOUNT {LOTPRICE} [LOTDATE] (LOTNOTE).

In Beancount

Beancount 2: https://beancount.github.io/docs/how_inventories_work.html Uses @ and {...}. It's still complicated, see https://www.reddit.com/r/plaintextaccounting/comments/ltqirt/one_lot_per_tx_or_how_to_handle_cost_basis_for

Beancount 3 (proposed): http://furius.ca/beancount/doc/v3-booking (uses @ with a bit of {...} mainly for backward compatibility)

In beans

Transaction prices are not declared explicitly, but inferred for every posting not already in the target commodity, from the market prices at posting date. With these, unrealized gains are calculated.

There's no way to select lots being sold, and realized gains are not calculated (?).

See also #1015#related-docs.

In lotter

It reads a journal of just the basic transactions and costs, and writes new journal entries enriched with lot info (?)

"lotter has a good approach because it separates the "what happened" from "what does it mean?"."

In hledger

Presently:

hledger doesn't have a special concept for lots, you model them manually with subaccounts. The close command and a powerful text editor are useful for constructing journal entries.

In future:

Any multi-commodity transaction (with explicit or inferred transaction price) ?

An explicit transaction price is written (@ or @@) ?

A transaction-priced increase to assets transferred from anywhere (eg revenue) ?

A transaction-priced increase to assets transferred from another asset ?

Some special syntax to activate lot tracking ?

Internally, when to use lots rather than ordinary amounts ?

Lots for everything ?

Lots for postings with a transaction price ?

Lots for postings of a commodity which has a known market price on the posting date ?

How to represent lots internally ?

Eg:

-- | An amount of some commodity transacted at a certain price on a certain date. 
data Lot = {
   lamount   :: Amount
  ,lprice    :: Price
  ,ldate     :: Day     -- ^ same as posting date
  ,lsequence :: Int     -- ^ sequence number ? (within lots of this commodity at this price on this day)
  }

-- | Lots in date and intra-day order, separated by commodity.
type Lots = Map CommoditySymbol [Lot]

data Account = Account {
   aname                     :: AccountName    -- ^ this account's full name
  ...
  ,aebalance                 :: MixedAmount    -- ^ this account's balance, excluding subaccounts
  ,aibalance                 :: MixedAmount    -- ^ this account's balance, including subaccounts
  ,alots                     :: Lots           -- ^ any priced lots held in this account
  }

Would alots equal aebalance, or augment it, or replace it ?

But actually every Amount already contains a (possibly empty) Price, used to represent transaction price. We could add the date (and sequence number?) there. Then Amount could represent an ordinary amount or a lot, and MixedAmount could contain a mixture of amounts and lots.

Is storing date in Amount redundant with the Posting's date ?

We think no, these could be different - see the stock split example below.

When a posting has both a transaction price and a market price, which one is used as lot price ? Which one is used to calculate capital gains ?

I think the transaction price. Ie the price you paid, not the prevailing market price on the day.

What are some basic ways of showing lots ?

• 1018 Ledger-style balance --lots, register --lots reports

• print ?

• hledger close --close --show-costs ACCTS

Is revaluation due to market price changes, different from unrealized capital gains ?

I think not. More discussion on #1029.

[alensiljak]

I would add to your last sentence and argue that adding negative lots is also a deposit. 😄 What I mean is, perhaps it would be possible to implement the simple way the ledger does it when no lots are matched on sale. In that case negative lots are added. The lot information looks absolutely crazy as there are positive and negative positions but the balance is correct in the end. This is how my book looks like after the import from GnuCash. Something like this might be an early implementation. I might provide some illustrated examples if you wish.

[simonmichael]

Thanks! Too much to digest right now, could revisit when we get a bit further.

[alensiljak]

Storing date in Amount is redundant with the Posting's date, but convenient when doing arithmetic with amounts/lots without their postings.

If by "Amount" you mean storing date as a part of the lot metadata, then that is not really redundant. The repricing operation shows that the transaction date != lot date.

2019-01-01 Buy some commodity
    Assets:Commodity    10 COM @ 10 EUR
    Assets:Cash           -100 EUR

2019-02-01 Reprice for whatever reason (stock split?)
    Assets:Commodity    -10 COM {10 EUR} [2019-01-01] 
    Assets:Commodity      5 COM {20 EUR} [2019-01-01]

The transaction date 2019-02-01 is just a date when this event happened. The lot date, however, must be 2019-01-01 for potential future tax liability. I think the taxation is the only issue here, though.

[simonmichael]

Helpful example - thanks! That seems right.

[alensiljak]

I see I've used "cost" when I meant "date". Too tired. Corrected.

[the-solipsist]

Some special syntax to activate lot tracking ?

I don't have a well-informed opinion about how to go about this issue in general, but I feel a special syntax ought to be supported.

Rationale: I use the "trading accounts" method of converting between commodities, rather than using hledger's built-in options (like @ or @@). Some of the commodities I have are investments that need to be tracked by lots since they incur capital gains tax depending on how long they are held before the gains are realized. Given that, I would like some way of indicating that specific kinds of commodity purchases/sales ought to be tracked as lots.

[simonmichael]

Thanks for the input @the-solipsist. Are you saying that lots should not be tracked with ordinary @/@@ usage, but only when explicitly requested by a new syntax ?

[the-solipsist]

Thanks for the input @the-solipsist. Are you saying that lots should not be tracked with ordinary @/@@ usage, but only when explicitly requested by a new syntax ?

Nope. I was thinking about explicit declaration of lots as an important fallback option for situations when clever auto-detection techniques don't work. And perhaps there is a clever auto-detection technique for the usage of trading accounts too, though I haven't given it enough thought. I also haven't really considered the implications of something getting declared as a lot even though it needn't be (i.e., in situations when lot-specific features aren't used by the user in relation to a transaction that hledger is tracking as a lot). I don't think there should be any negative consequences. Given this, perhaps auto-detection techniques for lots can err on the side of over-inclusion rather than under-inclusion. Or maybe there are some (even obvious) negative consequences that I'm missing.

[simonmichael]

Yes.. we need to come up with a clear design/spec to move this forward. I am not at all clear on it myself right now, and I'm more focussed on #1029 next.

[simonmichael]

Some things that would be helpful here:

• a clearer statement of the goal of this issue. Above it says "adding lots to an account, and reporting that they are there". I guess that needs elaboration. Eg which accounts can hold lots ? How do we report the existence of lots ? How are lots different from hledger's current functionality ?
• more research on other ledger-likes, in particular beancount, to see if they have a relevant feature that can serve as a good model. I think Ledger and beans do not, though feel free to explain otherwise.

[alensiljak]

I'm more focussed on #1029 next

That doesn't seem likely without lots? Lots are just sets of (date of purchase, quantity, price, currency) i.e. 2019-06-23, 18 units of Sugar at A$2.39 To get to un/realized gains/losses, one needs to know the lot information to calculate it, right? We take the price on another day, 2019-06-24, Sugar = A$2.40 and calculate 18 units * price difference.

Albeit, yes, there is already part of this implemented, so it does warrant a more specific requirement. BTW, what do you think is missing in Ledger's implementation? I find it quite practical in that it covers all my scenarios which involve investments. It would be convenient to have FIFO matching, which is the default in quite a few countries, by default but it's not a big deal.

[alensiljak]

Here's my shot at it:

a clearer statement of the goal of this issue. Above it says "adding lots to an account, and reporting that they are there". I guess that needs elaboration. Eg which accounts can hold lots ? How do we report the existence of lots ? How are lots different from hledger's current functionality ?

For one, we should be able to visualize the lots - list them, with related information (dates, quantities, commodities, prices). Basically, ledger bal with --lots. I use this frequently for sales, to match the units being sold. Having automatic FIFO matching would make lot information merely a nuissance for the user but they would still be the underlying base for the calculation of gains/losses.

The lot's lifespan is from purchase until the whole lot has been sold. It cannot be added to even though, as in the cast of repricing, it can be "overwritten". Ref: https://github.com/simonmichael/hledger/issues/1022#issuecomment-492284832

[adept]

I tried to find it, but my google-fu is no strong enough today :(

Is there any support in ledger for corporate actions (things that change quantity of shares you own/ your cost basis)? For example, N:M stock split means that every N shares you had become M shares, and share price becomes price*N/M (and you need to take it into account if you are doing FIFO accounting)

[alensiljak]

Yes, see my comment https://github.com/simonmichael/hledger/issues/1022#issuecomment-492284832. Credits to Martin Michlmayr, https://groups.google.com/forum/m/#!topic/ledger-cli/Y9jwyiBi97E

[adept]

I read the link but I dont think that I see example of corporate action handling there. Can you please link the exact comment from there?

[alensiljak]

Could be the app I'm using on the tablet. Here's the link from the browser: https://github.com/simonmichael/hledger/issues/1022#issuecomment-492284832

The comment from May 14

[the-solipsist]

I found this document by Martin Blais (@blais) to be very useful, as it provides a clear design spec as well, along with tons of context and clear thinking about the problem.

[alensiljak]

Perhaps Ledger was updated since that document but I find that most of the listed shortcomings spring either from unawareness of existence of some features or simply the document not being updated to track the software changes. After having retrofitted existing data with Ledger lot syntax, I've covered all cases that I ran into in the past. The lot implementation is quite OK. Additionally, having an automatic FIFO principle would eliminate most, if not all, manual tracking on the side of the user. One of the difficulties in manually matching lots is that there is no way (that I know of) of displaying the full lot information, sorted by date (https://github.com/ledger/ledger/issues/1802).

[the-solipsist]

The lot implementation is quite OK.

Going by this thread on the ledger-cli mailing list, I wouldn't say so.

[blais]

Does ledger now offer any support for matching lots? E.g. if you sell a lot you did not buy - from a data entry mistake - does it output an error?

On Wed, Jul 3, 2019, 15:31 Alen Siljak notifications@github.com wrote:

Perhaps Ledger was updated since that document but I find that most of the listed shortcomings spring either from unawareness of existence of some features or simply the document not being updated to track the software changes. After having retrofitted existing data with Ledger lot syntax, I've covered all cases that I ran into in the past. The lot implementation is quite OK. Additionally, having an automatic FIFO principle would eliminate most, if not all, manual tracking on the side of the user. One of the difficulties in manually matching lots is that there is no way (that I know of) of displaying the full lot information, sorted by date ( ledger/ledger#1802 https://github.com/ledger/ledger/issues/1802).

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/simonmichael/hledger/issues/1022?email_source=notifications&email_token=AACSBE5NNZ2KR3CS33RHOC3P5RIWHA5CNFSM4HMSPQU2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODZDRVYA#issuecomment-507976416, or mute the thread https://github.com/notifications/unsubscribe-auth/AACSBE4VNTMT5S5UA3B2LQTP5RIWHANCNFSM4HMSPQUQ .

[alensiljak]

Going by this thread on the ledger-cli mailing list, I wouldn't say so.

I'm basing my opinion on my own experiences, not on the forum postings. It has, and continues to work well for all the cases I've had in the past 20 years. That's good enough for me. Unless you can list a concrete example, or better yet, try it out with ledger, there's not much point discussing it in such a way. After wasting several minutes reading the referenced thread, I could not identify any relevant information there. I did notice, though, a few questions without answers and people who claim that they "don't have experience using lots in ledger". That puts the value of information in that thread way down the scale.

Martin, "selling lots you did not buy" will show a red negative lot, i.e. "-5 AAPL @ 15 USD" when lots are listed. It does not output an error because, as was mentioned many times, it does not impose its rules on the users the way Beancount does, which is also the main reason I did not manage to convert my data to Beancount after several weeks of trying. I prefer flexible systems. A red negative lot, when I don't expect one, is a good-enough warning sign to me.

[blais]

On Fri, Jul 12, 2019, 17:16 Alen Siljak notifications@github.com wrote:

Going by this thread on the ledger-cli mailing list, I wouldn't say so.

I'm basing my opinion on my own experiences, not on the forum postings. It has, and continues to work well for all the cases I've had in the past 20 years. That's good enough for me. Unless you can list a concrete example, or better yet, try it out with ledger, there's not much point discussing it in such a way. After wasting several minutes reading the referenced thread, I could not identify any relevant information there. I did notice, though, a few questions without answers and people who claim that they "don't have experience using lots in ledger". That puts the value of information in that thread way down the scale.

Martin, "selling lots you did not buy" will show a red negative lot, i.e. "-5 AAPL @ 15 USD" when lots are listed. It does not output an error because, as was mentioned many times, it does not impose its rules on the users the way Beancount does, which is also the main reason I did not manage to convert my data to Beancount after several weeks of trying. I prefer flexible systems. A red negative lot, when I don't expect one, is a good-enough warning sign to me.

TLDR (and I'm on vacation) but what you describe only "works" because it doesn't do what it should, which is to match the lots against the inventory at the date they get sold, not at the date the balance sheet gets computed. Beancount's design isn't too impose a rule at my whim but rather simply to implement a system with the minimum amount of configuration options that does the "right thing" and automatically brings up a maximum amount of legitimate data entry errors. If you failed to convert you probably still have conversion errors (or there is a bug). You're just choosing to live with incorrect data, by using a system that doesn't detec... Huh, I mean a system that is flexible. A spreadsheet is even more flexible btw.

Your file probably contains errors. Have fun,

—

You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/simonmichael/hledger/issues/1022?email_source=notifications&email_token=AACSBE2QILKZZBQWPLRTY3LP7A4ULA5CNFSM4HMSPQU2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODZZB7XY#issuecomment-510795743, or mute the thread https://github.com/notifications/unsubscribe-auth/AACSBE3CDXOZNKT7K5YCBP3P7A4ULANCNFSM4HMSPQUQ .

[the-solipsist]

Unless you can list a concrete example, or better yet, try it out with ledger, there's not much point discussing it in such a way.

The mail that I linked to shows a concrete example with sample files, where --unrealized doesn't work as it should, and where {} and @, which should (according to the manual) be interchangeable in ledger syntax for lots, but the syntaxes produce different results. While I'm glad your experience with ledger's lot handling has been positive, I'm not sure that is the same as ledger's implementation of lots being OK.

[alensiljak]

--unrealized indeed have some issues but I can't comment on that as I saw it looked weird in some cases and had no time to set up some specific scenarios to find out why. I have not really had the need to use that option. Perhaps my understanding of how it should work is wrong, perhaps there is a bug in the code. The same applies to other posters in the thread but we can agree that the results are not what they would expect to see.

What I see in the scenario, as far as lots go, nowhere did the user specify which lot gets sold. He merely puts

2019/02/02 Broker
    Assets:Stocks                                 -1 AAPL @ $540.00 

expecting what? Magic? There is no FIFO nor any other lot-matching scheme in ledger, from what I know. And, quite correctly, he gets the following results

19-Feb-02 Broker            Assets:Stocks              -1 AAPL {$540}            $-1000
                                                        2 AAPL {$500}
                                                       -1 AAPL {$540}

meaning he still has a lot of 2 AAPL @ 500 and a new lot of -1 AAPL @ 540. This is really not that hard to wrap the head around. On the other hand, one could also say it's a feature because it can be used to track short positions.

Had he specified which lot is being sold, the results would be correct and the specified lot would get reduced.

As I hinted, this allows flexibility to migrate years of data from other accounting systems and continue producing correct results while allowing one to correct the lot transactions on as-needed basis. I see it as a big plus since I migrated the data from GnuCash in two weekends and had a working system which still allows me to re-write the lot transactions and make them match exactly, when I need them, or just leave them as-is and still have the correct end balances for the historical transactions.

So, while there may be an issue with the calculation of unrealized gains/losses I still fail to see an issue with lots per-se. And I believe this is what we are discussing here.

[simonmichael]

The discussion is appreciated. Since this topic is so large and confusing, this issue describes carefully what it wants to clarify and achieve at the top. Let's try to extract insights that are relevant and actionable for this issue, and try not to get too sidetracked - for freewheeling discussion there is also the mail list. @MisterY you might have experience and understanding of Ledger's functionality that many of us have struggled to acquire. If you can supply better short answers for any of the Questions above, in a clear, concise reproducible manner with concrete examples, that's particularly useful.

[simonmichael]

Also helpful would be answers for https://github.com/ledger/ledger/issues/1794 (price syntax/--revalued/--gains mysteries), where I tried to document and understand some Ledger behaviour in this area.

[simonmichael]

Also, as I said above I see https://github.com/simonmichael/hledger/issues/1029 as a prerequisite/earlier milestone for hledger, so help with that one first is very welcome.

[alensiljak]

Thanks for emphasizing the constraints and dependent issues, @simonmichael. This is a broad topic, especially if we take into consideration the breadth of the topic. I'm here fairly narrow-minded and am assuming here compatibility with ledger journals, which affects my understanding of the implementation. The questions in the first post actually open the topic by listing the current implementations. Martin has written extensively about the implementation in Beancount, and I'm sure he could add more as the code has probably progressed since the docs were written. I do know a little bit about lots but am by no means an expert. From what I've seen so far, Martin Michlmayr is probably the most well-rounded source of knowledge across all PTA systems.

I am definitely looking to help with what I've learned so far and can also provide syntax examples for specific cases.

Additionally, I find this question confusing: "When a posting has both a transaction price and a market price, which one is used as lot price ? Which one is used to calculate capital gains ?" I'd assume one would have to be ditched. The @ price seems more specific than @@ and should be more precise as @@ amounts can be rounded, resulting in crazy @ prices.

"Internally, when to use lots rather than ordinary amounts ?" - whenever a lot syntax is used: @ price, @@ amount, [ date ]. (probably an incomplete list)

Should we try to do this as a wiki page with the proposal and discussion, perhaps?

[blais]

[sm: SNIP]

Additionally, I find this question confusing: "When a posting has both a transaction price and a market price, which one is used as lot price ? Which one is used to calculate capital gains ?"

The cost price can be used to identify which lot to reduce

The other postings should be used to compute the capital gains (note that commissions are a bit problematic)

The market price can be used to verify the proceed amount, see sellgains plugin

I'd assume one would have to be ditched. The @ price seems more specific than @@ and should be more precise as @@ amounts can be rounded, resulting in crazy @ prices.

@@ gets converted into @, it's as simple as that

[simonmichael]

I find this question confusing: "When a posting has both a transaction price and a market price, which one is used as lot price ? Which one is used to calculate capital gains ?" I'd assume one would have to be ditched. The @ price seems more specific than @@ and should be more precise as @@ amounts can be rounded, resulting in crazy @ prices.

@MisterY, @ and @@ are just different notations for specifying a transaction price.

Should we try to do this as a wiki page with the proposal and discussion, perhaps?

As you see I'm trying to (ab)use issues as a wiki page.. feel free to write/share notes elsewhere if more convenient. hledger's github wiki is kind of dedicated to the cookbook. Maybe a github gist ?

[div0man]

Hi all, I recently discovered ledger, and while researching the issue of tracking lots I discovered this handy tool, maybe it will give you some ideas: https://src.d10.dev/lotter/doc/tip/README.md

For my trial it worked really well. Using ledger-cli + lotter I was able to process all my stock trades and get the same result I had with manual spreadsheet accounting, and it took me only about an hour. Great stuff!

It does almost all I need although I had to modify the code a little because I'm not an US resident and my jurisdiction treats short-term/long-term gains differently.

I'm still thinking about how to "properly" account for trading fees/commissions because buying fees should be accounted for as a loss but at the time of sale. I can include the fees into buy cost basis for correct gains calculation and that works but then I lose track of how much fees I paid.

To track fees, virtual posting works but it's not perfect because if placed in a buy transaction it will be tracked at wrong date and in case of no selling or partial sells it will not show the correct total. Placing them in the sell transaction requires more manual labor but it is the correct way with the caveat that you have to go back in time to find the amount of fee.

There are also issues of cost basis transfer AKA "deferred gains" which is allowed in some jurisdictions. Lotter handles that well, but accepts only 1 currency as a gain-trigger, while it should really be any currency however due to ledger architecture there's no way to tell apart currencies from commodities so there's room for improvement there.

I think lotter has a good approach because it separates the "what happened" from "what does it mean?". Transactions happening at a certain price are facts regardless of gain-calculating method used so I guess the solution should reflect this reality. Have a way to record the facts (ledger), and have a way to compute them as required (lotter), i.e. FIFO, LIFO, manual override, average, with options for holding times, etc.

So there's food for thought, I hope you'll find it helpful!

[simonmichael]

Thank you very much, I will check it out!

[simonmichael]

@div0man, I've linked this on plaintextaccounting.org. FYI I'm having a hard time trying it out, with figuring out how to get a working copy of the fossil repo (fossil clone URL lotter.fossil; mkdir lotter; cd lotter; fossil open ../lotter.fossil) and how to build a go project; consider adding this to the readme.

[div0man]

@simonmichael Yeah that gave me a little trouble, too. Should have written the steps right away to save you the trouble, sorry.

1. Install go and fossil packages
2. Clone the repo mkdir lotter cd lotter fossil clone https://src.d10.dev/lotter lotter.fossil
3. Fossil is a little different than git so we need to unpack the files from the repo fossil open lotter.fossil
4. Automatically download all dependencies, they'll be placed in ~/go folder go get
5. Build the binary, it'll appear in the current folder go build

[adept]

I also had to do: mkdir ~/go/bin export GOBIN=~/go/bin (otherwise "go get" complained about GOPATH)

I get a failure at the end of "go get" about a bunch of redefined debug-related symbols and then it all stops ... I will poke around a bit more and see if I can have it built ...

../../go/src/src.d10.dev/command/log_wasm.go:25:20: Debug redeclared in this block previous declaration at ../../go/src/src.d10.dev/command/log.go:49:20 ../../go/src/src.d10.dev/command/log_wasm.go:26:36: Debugf redeclared in this block previous declaration at ../../go/src/src.d10.dev/command/log.go:50:36 ../../go/src/src.d10.dev/command/log_wasm.go:27:19: Info redeclared in this block previous declaration at ../../go/src/src.d10.dev/command/log.go:51:19 ../../go/src/src.d10.dev/command/log_wasm.go:28:35: Infof redeclared in this block previous declaration at ../../go/src/src.d10.dev/command/log.go:52:35 ../../go/src/src.d10.dev/command/log_wasm.go:30:20: Error redeclared in this block previous declaration at ../../go/src/src.d10.dev/command/log.go:54:20 ../../go/src/src.d10.dev/command/log_wasm.go:36:36: Errorf redeclared in this block previous declaration at ../../go/src/src.d10.dev/command/log.go:60:36 ../../go/src/src.d10.dev/command/log_wasm.go:43:27: setLogBackend redeclared in this block previous declaration at ../../go/src/ src.d10.dev/command/log_release.go:27:27

On Fri, Jun 5, 2020 at 9:20 AM div0man notifications@github.com wrote:

@simonmichael https://github.com/simonmichael Yeah that gave me a little trouble, too. Should have written the steps right away to save you the trouble, sorry.

1. Install go and fossil packages
2. Clone the repo mkdir lotter cd lotter fossil clone https://src.d10.dev/lotter lotter.fossil
3. Fossil is a little different than git so we need to unpack the files from the repo fossil open lotter.fossil
4. Automatically download all dependencies, they'll be placed in ~/go folder go get
5. Build the binary, it'll appear in the current folder go build

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/simonmichael/hledger/issues/1022#issuecomment-639332812, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAA42KVPTPE55LKGHQ35ELTRVCTGLANCNFSM4HMSPQUQ .

-- D. Astapov

[adept]

This is how i built it:

Picking up after (fossil open lotter.fossil):

Start go docker container: docker run --rm -it -v $(pwd):/usr/src/myapp -w /usr/src/myapp golang:1.14 bash

In it, do "apt-get update && apt-get install fossil", then "export USER=root", "mkdir -p /go/bin", "export GOBIN=/go/bin", and then, finally, "go get" and "go build"

On Fri, Jun 5, 2020 at 9:17 PM Dmitry Astapov dastapov@gmail.com wrote:

I also had to do: mkdir ~/go/bin export GOBIN=~/go/bin (otherwise "go get" complained about GOPATH)

I get a failure at the end of "go get" about a bunch of redefined debug-related symbols and then it all stops ... I will poke around a bit more and see if I can have it built ...

../../go/src/src.d10.dev/command/log_wasm.go:25:20: Debug redeclared in this block previous declaration at ../../go/src/src.d10.dev/command/log.go:49:20 ../../go/src/src.d10.dev/command/log_wasm.go:26:36: Debugf redeclared in this block previous declaration at ../../go/src/src.d10.dev/command/log.go:50:36 ../../go/src/src.d10.dev/command/log_wasm.go:27:19: Info redeclared in this block previous declaration at ../../go/src/src.d10.dev/command/log.go:51:19 ../../go/src/src.d10.dev/command/log_wasm.go:28:35: Infof redeclared in this block previous declaration at ../../go/src/src.d10.dev/command/log.go:52:35 ../../go/src/src.d10.dev/command/log_wasm.go:30:20: Error redeclared in this block previous declaration at ../../go/src/src.d10.dev/command/log.go:54:20 ../../go/src/src.d10.dev/command/log_wasm.go:36:36: Errorf redeclared in this block previous declaration at ../../go/src/src.d10.dev/command/log.go:60:36 ../../go/src/src.d10.dev/command/log_wasm.go:43:27: setLogBackend redeclared in this block previous declaration at ../../go/src/ src.d10.dev/command/log_release.go:27:27

On Fri, Jun 5, 2020 at 9:20 AM div0man notifications@github.com wrote:

@simonmichael https://github.com/simonmichael Yeah that gave me a little trouble, too. Should have written the steps right away to save you the trouble, sorry.

1. Install go and fossil packages
2. Clone the repo mkdir lotter cd lotter fossil clone https://src.d10.dev/lotter lotter.fossil
3. Fossil is a little different than git so we need to unpack the files from the repo fossil open lotter.fossil
4. Automatically download all dependencies, they'll be placed in ~/go folder go get
5. Build the binary, it'll appear in the current folder go build

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/simonmichael/hledger/issues/1022#issuecomment-639332812, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAA42KVPTPE55LKGHQ35ELTRVCTGLANCNFSM4HMSPQUQ .

-- D. Astapov

-- D. Astapov

[div0man]

I tried to account for some stock splits and noticed that the date doesn't carry correctly. Here's an example:

2020-01-01 Equity
    Equity:Cash           -100.00 EUR
    Assets:Cash            100.00 EUR

2020-01-01 Buy AAA
    Assets:Cash           -100.00 EUR
    Assets:Stocks            1.00 AAA @@ 100 EUR

2020-02-01 Temporary commodity to hold the value
    Assets:Stocks           -1.00 AAA
    Assets:Stocks:Splits     1.00 TEMP @@ 1 AAA

2020-02-01 Stock split
    Assets:Stocks:Splits    -1.00 TEMP
    Assets:Stocks           10.00 AAA @@ 1 TEMP

P 2020-03-01 00:00:00 AAA 20 EUR
2020-03-01 Sell AAA
    Assets:Stocks           -5.00 AAA @@ 100 EUR
    Assets:Cash            100.00 EUR

Running lotter -f test.txt -base EUR lot | ledger -f - bal -X EUR -E I get:

          200.00 EUR  Assets
          100.00 EUR    Cash
          100.00 EUR    Stocks
                   0      Splits
         -100.00 EUR  Equity:Cash
         -100.00 EUR  Lot
          -50.00 EUR    
                   0      2020/01/01
                   0        1AAA@100EUR
                   0        1TEMP@1AAA@100EUR
          -50.00 EUR      2020/02/01:10AAA@0.1TEMP@100EUR
          -50.00 EUR    Income:short term gain
--------------------
                   0

While it demonstrates the convenience of using lotter as a pre-processor, it produces wrong output for my use-case. Too bad :(

It correctly shows that I started with 100 EUR, then bought 1 AAA, which got split 10-for-1 into 10 AAA, later appreciated +100% at which point I took some gains, and am still holding 5 AAA worth 100 EUR with unrealized gains shown under that lot's account.

However, the date got messed up. Looking into line 698 of op_lot.go I understand that lotter carries forward the latest date of consumed inventory. This can introduce weird behavior, especially if multiple lots are consumed in a single trade.

I think it should either preserve the oldest date(s) or set the date of newly created lot to the transaction date, so 2 modes of operation.

If old dates are to be carried forward, then lotter should create a new lot for each different input date, and can't merge them into a single one as it does now.

This is something I should share with the author of lotter but I figure it's relevant here, too.

PS Having multiple currencies further complicates things, and my approach was to create separate ledger files for different currencies and run lotter on each with different --base setting prior to merging them and feeding into ledger.

[simonmichael]

I'm glad to see the logic and UX being worked on, this will be very helpful for adding similar functionality to hledger.

[div0man]

There was a little bug causing that weird behavior with dates, so if anyone wants to use lotter, I advise to build it with this fix: https://github.com/div0man/lotter/commit/6ef3cdb0f25766d6a5432f8220601d687d1ad9ed

There's still the issue / missing feature of consuming multiple lots in a single trade. Right now, you'd have to make a separate trade for each lot in order to preserve lot dates and cost basis.

I also contacted the original author, and let him know about it and that I shared his software here. Don't know whether he reads the e-mail I used, because it seems like he intentionally removed contact details, and I dug up his e-mail from some forgotten copyright file :)

[dncohen]

lotter author here, hi.

I intended the install to be as simple as go get src.d10.dev/lotter. Sorry for any inconvenience there.

Based on a quick scan of div0man's issue, I think it is a good catch. I wonder if I have to refile some old taxes!

[simonmichael]

$65 bounty contributed by @pkel on Bountysource.

[lestephane]

$65 bounty contributed by @pkel on Bountysource.

Trying to parse what that means. Is there a BountySource bounty whose goal is to fix this issue? If so, what is the link to it?

[simonmichael]

Yep; I added the link to the issue's description only (see top of https://github.com/simonmichael/hledger/issues/1022).

[simonmichael]

More context: it took us a while to figure out Bountysource again. Reactivating its integration with this site requires too much access to my github orgs (and it was buggy last time I used it), so it's just manual integration for now.

[simonmichael]

Possibly related discussion: https://www.reddit.com/r/plaintextaccounting/comments/ltqirt/one_lot_per_tx_or_how_to_handle_cost_basis_for

[sbrow]

I would very much like to solve this issue by implementing the beancount inventory system (as described here: https://beancount.github.io/docs/how_inventories_work.html in a fork. Would you be receptive to this @simonmichael ?

I'm not particularly fond of the curly brace syntax, but syntax can always be changed.

[blais]

If for whatever reason you do some work to revise your matching system, you might be interested in the improvement I'm proposing to implement for the next version, which I think simplifies things a lot and is more sensible than what we have now: https://docs.google.com/document/d/1H0UDD1cKenraIMe40PbdMgnqJdeqI6yKv0og51mXk-0/edit#heading=h.q5k7iktofra4

On Sun, Mar 27, 2022 at 8:33 PM Spencer Brower @.***> wrote:

I would very much like to solve this issue by implementing the beancount inventory system (as described here: https://beancount.github.io/docs/how_inventories_work.html in a fork. Would you be receptive to this @simonmichael https://github.com/simonmichael ?

I'm not particularly fond of the curly brace syntax, but syntax can always be changed.

— Reply to this email directly, view it on GitHub https://github.com/simonmichael/hledger/issues/1022#issuecomment-1080059172, or unsubscribe https://github.com/notifications/unsubscribe-auth/AACSBE6EDWQXDPTKO5MD433VCD44RANCNFSM4HMSPQUQ . You are receiving this because you were mentioned.Message ID: @.***>

[simonmichael]

Thank you both, I'm very interested in any experiments or mockups you might come up with @sbrow, especially if it involves Martin's latest designs. I'll do some reading to get up to speed.

[simonmichael]

I did a first read of http://furius.ca/beancount/doc/v3-booking (@blais's link above). This might not be the right place, but quick comments: Martin I'm happy that you too feel @ is sufficient, and that {...} is being de-emphasised. And I wonder what you have thought of using another existing mechanism, subaccounts, to keep track of lots ?

[simonmichael]

For example, here's a trade in beancount v2 syntax:

2020-11-02 * "Broker" "Plan Transfer"
  Assets:Investments:Broker:AAAAA  -25.74980 AAAAA {} @ 30.19 USD
  Assets:Investments:Broker:BBBBB   40.00960 BBBBB {19.43 USD}
  Income:Broker:Capital-Gains:AAAAA

expressed with subaccounts instead of {...} notation (hledger example, since I have it here to test):

2020-11-02 * Broker | Plan Transfer
    Assets:Investments:Broker:AAAAA:2020-06-01-30.19-USD  -25.74980 AAAAA @ 30.19 USD
    Assets:Investments:Broker:BBBBB:2020-11-02-19.43-USD   40.00960 BBBBB @ 19.43 USD
    Income:Broker:Capital-Gains:AAAAA                      -0.00007 USD

Optional conveniences: one could indicate lot accounts with :: if needed, auto-infer them when acquiring something with a cost, auto-infer cost basis and a gain/loss posting when disposing from a lot account, etc. Eg the above could be inferred from:

2020-11-02 * Broker | Plan Transfer
    Assets:Investments:Broker:AAAAA::2020-06-01-30.19-USD  -25.74980 AAAAA  ; select the lot by name
    Assets:Investments:Broker:BBBBB                         40.00960 BBBBB @ 19.43 USD
    Income:Broker:Capital-Gains

or:

2020-11-02 * Broker | Plan Transfer
    Assets:Investments:Broker:AAAAA   -25.74980 AAAAA @ 30.19 USD  ; select the lot by cost basis
    Assets:Investments:Broker:BBBBB    40.00960 BBBBB @ 19.43 USD
    Income:Broker:Capital-Gains

or:

2020-11-02 * Broker | Plan Transfer
    Assets:Investments:Broker:AAAAA   -25.74980 AAAAA                  ; select the lot by FIFO
    Assets:Investments:Broker:BBBBB    40.00960 BBBBB @ 19.43 USD
    Income:Broker:Capital-Gains