stellar / go

Stellar's public monorepo of go code
https://stellar.org/developers
Apache License 2.0
1.31k stars 499 forks source link

Implement function which merges two LedgerCloseMeta objects #5479

Open tamirms opened 2 months ago

tamirms commented 2 months ago

We want a function which takes two LedgerCloseMeta xdr objects and returns a LedgerCloseMeta object which includes all the transactions and meta from the two source LedgerCloseMeta xdr objects. The purpose of this function is to create ledgers with arbitrarily an large number of transactions. Instead of generating fake transactions and meta we can combine existing ledgers to create large ledgers.

tamirms commented 2 months ago

I prototyped the merge function here . However, I ran into a problem related to the specific order in which ledger entry changes are applied.

Changes within a ledger should be applied in the following order:

  1. fee changes of all transactions
  2. transaction meta changes of all transactions
  3. upgrade changes

So if we have two ledgers the order of application for those two ledgers will be:

  1. fee changes of all transactions from ledger 1
  2. transaction meta changes of all transactions from ledger 1
  3. upgrade changes from ledger 1
  4. fee changes of all transactions from ledger 2
  5. transaction meta changes of all transactions from ledger 2
  6. upgrade changes from ledger 2

However, if we try to merge ledgers 1 and 2 into a single ledger we can no longer maintain the ordering above and instead we will obtain the following order:

  1. fee changes of all transactions from ledger 1
  2. fee changes of all transactions from ledger 2
  3. transaction meta changes of all transactions from ledger 1
  4. transaction meta changes of all transactions from ledger 2
  5. upgrade changes from ledger 1
  6. upgrade changes from ledger 2

Given the importance of having the correct application order, I think we cannot proceed with combining several ledger objects into one. Instead, I think we'll need to refactor ingestion to support ingesting multiple ledgers within a single DB transaction as an approximation of having one large ledger which is the union of a bunch of smaller ledgers.