search

stellar / go

Stellar's public monorepo of go code
https://stellar.org/developers
Apache License 2.0
1.3k stars 499 forks source link

ingest/cdp: get latest ledger as default for unbounded range #5495

Open sreuland opened 3 weeks ago

sreuland commented 3 weeks ago

What problem does your feature solve?

for uses cases of cdp.ApplyLedgerMetadata where caller just wants it to start a range at whatever is latest ledger of network, requires the caller to obtain the latest ledger manually from history archives first and pass that as range.from

What would you like to see?

When caller of cdp.ApplyLedgerMetadata passes UnboundedRange.from=0, cdp.ApplyLedgerMetadata should detect and acquire the latest ledger internally and substitute that in the UnboundedRange.from value passed to BufferedStorageBackend.

An implementation choice needs be determined on how/which 'latest' ledger is obtained: 1) the checkpoint ledger from target network based on history archives 2) the greatest ledger sequence that exists on the Datastore

If second option is chosen to use Datastore, then this ticket does become effectively blocked on https://github.com/stellar/go/issues/5498 which provides the changes on Datastore to facilitate.

What alternatives are there?

callers of cdp.ApplyLedgerMetadata that want to start streaming from current network ledger, must manually call HistoryArchive.GetLatestLedgerSequence() first to obtain the latest sequence.

sreuland commented 2 weeks ago

It would be extra cool if Datastore could be queried to provide its highest and lowest ledger sequences currently available on it, then this type of behavior for clients to initiate unbounded from latest data point would be trivial: https://github.com/stellar/go/issues/5498