Sharding the world ledger

[michielbdejong]

I united the AccountHistoryChunk and MultiAccountView classes into one WorldLedgerView class. But the mutations are still a flat array. Should shard that by from and to, to define Shard which is basically the same as the HalfLedger from SNAP. But it's not a "half" thing because this is the world ledger and there are way more than two shards.

Within a shard, we should have smart mechanisms to deal with date. There can be periods for which the shard is exhaustive.

Ah so the exhaustiveChunks list should also move from https://github.com/solid/money-pane/blob/bcd5dfb/src/Ledger.ts#L230 into the individual shards.

[michielbdejong]

Another way to deal with exhaustive chunks would be the inverse - dark stretches that enter as entries in the mutations list. If you have no exhaustion then there would be a dark stretch inbetween each two mutations. maybe a flag would be enough to say whether or not a mutation is preceded by a dark stretch or not (previous = null).

[michielbdejong]

Hm, maybe we don't necessarily want to give previousMutation a place inside WorldLedgerMutation, and intermixing WorldLedgerMutation | DarkStretch elements into a single array would lead to lots of if-statements when looping through that array. If we have a separate array of DarkStretch objects (not mixed in with the mutations) then we may as well have an array of ExhaustiveChunk as we have now.

[michielbdejong]

Exhaustiveness for group ledgers like WieBetaaltWat is also interesting - if there is a group ledger 'drinks after work' and on there is a transaction 'alice -> bob [1 beer]', then we should import that into a alice@drinks-after-work -> bob@drinks-after-work shard (not a alice - > bob shard) and say it's exhaustive because we know our export from the group ledger is exhaustive.

You could also model a non-exhaustive alice -> bob shard, that has an exhaustive sub-shard alice@drinks-after-work -> bob@drinks-after-work.

[michielbdejong]

So a shard should have:

• from (account@system-of-record)
• to (account@system-of-record)
• exhaustiveStretches
  • startDate
  • startDateIsInclusive
  • endDate
  • endDateIsInclusive
  • startBalance?
  • amount
  • unit
• mutations (could leave out from and to but let's not go there)

What should you do when two identical mutations are reported outside an exhaustiveStretch? Probably default to merging them.

[michielbdejong]

Or, staying closer to what we have, leave mutations as an array, but add a from and a to to exhaustiveChunks.

[michielbdejong]

The mixIn function gets very complex if we don't do it at the Shard level

[michielbdejong]

Ah, but no! I made a mistake. A bank statement reports exhaustively about the edge between the customer and the bank. Transfers between bank accounts are additional mutations that are only implied. We can't know the startBalance of the ledger between two bank accounts. We can only know the startBalance between customer and bank. So if an expense is implied, then that is a third mutation, not a second one. 1: customer -> bank 2: bank -> shop 3: shop -> budget

[michielbdejong]

We can demand to always have a start balance from an import. We can provide an 'end balance to start balance' utility function for when you import your latest transactions and only know your current balance. We can keep using AccountHistoryChunk for this. But maybe split the account identifier into 'accountHolder' and 'bank'.

And maybe split it up into from:accountHolder to:bank and from:bank to:accountHolder? So there are only positive amounts? That would make it easier to then model bankAccount -> bankAccount transactions as well.

[michielbdejong]

Or when modeling bankAccount->bankAccount we just pick the to and from at random (e.g. alphabetically)

[michielbdejong]

or maybe even generate it on the fly from the chunks we have. a good next thing to do would be to go a step back to cfda6de53bf88734ae10df33c83bff5cd28a81d4 and debug the full multi-account import.