Database has a new table, called txhash2txnum which is basically a glorified multi-hash-map whereby the key is 6 bytes from the txhash and the buckets are a list of VarInt, each entry corresponding to an offset in the txnum2txhash flat file. This allows us to lookup tx's by txid and figure out if they are confirmed or not and if so at what block height.
This new index enables us to add a new facility: blockchain.transaction.subscribe, which resolves issue #32 .
This new index also allows us to offer a new RPC: blockchain.transaction.get_height, so that clients can find out if a txid is confirmed, and if so, in which block. A height of 0 is returned if the tx is in the mempool. A height of null is returned if the tx is unknown.
The RPC method blockchain.transaction.get_merkle now no longer requires the second height argument (it can still be specified and it does make the RPC slightly faster in that case). Fulcrum now knows the confirmed height of any tx if it exists in the blockchain, so the second argument is just a performance hack and now is no longer required.
Performance and disk space implications:
This new index eats about 1GB on testnet3 and between ~3 - ~5GB or so on mainnet (the variability in how much it eats is due to the way rocksdb works). The index doesn't eat too much time in terms of the addBlock pipeline for synching -- rocksdb has incredibly fast inserts. Even so, we added a new CoTask which doles the work out in parallel in the addBlock function to add to this index in another worker thread while we are busy processing the block in our Controller thread.
Migration path & compatibility:
This index is implemented as a separate rocksdb database, which lives in the datadir under txhash2txnum/.
Servers will automatically build this index on startup from the existing txnum2txhash flat-file. Building it on testnet3 with an SSD takes ~1 minute and on mainnet from between 3 and 10 minutes, depending.
If downgrading from this version back to the 1.4.x or earlier series -- the new database is simply ignored by older versions. Admins can then later move back up to later Fulcrum and the index will be detected as "stale" on startup and rebuilt from scratch on first run.
Changes:
txhash2txnumwhich is basically a glorified multi-hash-map whereby the key is 6 bytes from the txhash and the buckets are a list ofVarInt, each entry corresponding to an offset in the txnum2txhash flat file. This allows us to lookup tx's by txid and figure out if they are confirmed or not and if so at what block height.blockchain.transaction.subscribe, which resolves issue #32 .blockchain.transaction.get_height, so that clients can find out if a txid is confirmed, and if so, in which block. A height of 0 is returned if the tx is in the mempool. A height ofnullis returned if the tx is unknown.blockchain.transaction.get_merklenow no longer requires the secondheightargument (it can still be specified and it does make the RPC slightly faster in that case). Fulcrum now knows the confirmed height of any tx if it exists in the blockchain, so the second argument is just a performance hack and now is no longer required.Performance and disk space implications:
This new index eats about 1GB on testnet3 and between ~3 - ~5GB or so on mainnet (the variability in how much it eats is due to the way rocksdb works). The index doesn't eat too much time in terms of the
addBlockpipeline for synching -- rocksdb has incredibly fast inserts. Even so, we added a newCoTaskwhich doles the work out in parallel in theaddBlockfunction to add to this index in another worker thread while we are busy processing the block in ourControllerthread.Migration path & compatibility:
This index is implemented as a separate rocksdb database, which lives in the datadir under
txhash2txnum/.Servers will automatically build this index on startup from the existing
txnum2txhashflat-file. Building it on testnet3 with an SSD takes ~1 minute and on mainnet from between 3 and 10 minutes, depending.If downgrading from this version back to the 1.4.x or earlier series -- the new database is simply ignored by older versions. Admins can then later move back up to later Fulcrum and the index will be detected as "stale" on startup and rebuilt from scratch on first run.