aasseman
commented
1 year ago TAP updates the available collateral at every received receipt in memory, but if we split TAP between multiple indexer-service instances, there is no reasonable way to do those real-time updates. Here are a few solutions that we are considering:
- Give up on having the possibility of HA indexer-service, require 1 instance max per indexer. Perhaps with the Rust version, leveraging higher perf and possible multi-threading, the impact could be softened. Still a bit annoying IMO.
- Give up on having indexer-service instances keep an exact accounting of remaining collateral. They can update the remaining collateral at each RAV (we could sync new RAVs to all instances using PG's notify/listen, it seems). Each indexer-service would just try not to overdraft:
- ... considering its own observation of receipts since latest RAV. Some risk of overdraft, but bounded by the total amount in receipts in between RAVs.
- or ... considering only ((on-chain collateral) - (latest RAV amount)) / (# of index-service instances) collateral until the next RAV update. No risk of overdraft, but risk of false positives. Plus we'd need to know how many instances are running.
- Using Redis to store the collateral amount outside indexer-service instances. The instances would all subtract from that shared value. Probably the best compromise of simplicity and performance for sharing a single value, but very likely to still have a performance impact.
Scenario:
indexer-serviceinstances run their own manager to receive the receipts, do initial checks and store into a DBindexer-agent, or a new one dedicated to this) manages the final receipt checks (pulling from DB) and RAV requests.