Txindex

[D-Stacks]

Merge Request for TxIndex

Summary

This PR introduces the TxIndex, a new index that enables querying of transaction data.

Changes:

(None exhaustive)

Consensus Codebase

• Expands the get_virtual_chain_from_block API to allow chunked calls and walking the chain to a specific hash.
• Adds public APIs for retrieving block transactions, history root, highest common chain block ancestor, chain block status, and compact header.
• Optimizes SinkBlueScoreChanged, VirtualDaaScoreChanged, and VirtualChainChanged notifications to only emit when relevant data changes.
• Introduces a ChainAcceptanceDataPruned notification emitted by the pruning processor.
• Modifies VirtualChainChanged to include removed acceptance data.
• Moves CachePolicyBuilder to the kaspa-database crate and makes some functions public.
• Converts Testconsensus::with_db to use a dummy sender.
• Refactors test_helpers to testutils and adds generators for acceptance data.
• Moves CompactHeader to the core crate.
• Renames AcceptedTxEntry to TxEntry to accommodate for future support for unaccepted transactions.

Notify Codebase

• Add relevant parts for ChainAcceptanceDataPruned

Index Codebase

• Moves all core TxIndex and UTXOIndex structures to the core crate.
• Configures the index processor to register for ChainAcceptanceDataPruned and VirtualChainChanged notifications if the TxIndex is activated.
• Updates the UTXOIndex to also receive a notification in its update method for consistency in the processor.
• Integrates new notifications into the TxIndex in the processor.
• Adds the TxIndex itself, including relevant stores, reindexer, API, minimal configuration with memory-aware caches and resyncing.

Kaspad Codebase

• Enables indexes to initialize and resync in parallel using threads.
• Adds relevant database directories and setup for the TxIndex.
• Adds flags and arguments for configuring the TxIndex.

RPC Codebase

• Integrates the TxIndex into the RPC framework.
• Add new TransactionData structure for querying transactions.
• Adds code for a getTransactionData RPC call in both gRPC and wRPC and into ./kaspa-cli

Integration Tests Codebase

• Adds TxIndex-related tests to the RPC calls tests.
• Adds TxIndex-related tests to the JSON tests.

Known Issues

1. There is a race condition or similar issue when pruning is interrupted, which causes the TxIndex history root to lose sync with consensus. - Solved

2. There is a lack of testing for pruning and resyncing due to limitations in the testing framework. A separate PR may be needed to expand Simpa to support testing with indexes and enable controlled testing of reorg scenarios.