search

bitcoinjs / indexd

An external bitcoind index management service module
ISC License
53 stars 23 forks source link

Using `decoderawtransaction` instead of decoding through bitcoin.js #10

Closed karelbilek closed 7 years ago

karelbilek commented 7 years ago

Just a fast question

Would it be possible to use decoderawtransaction RPC call to decode transaction instead of using bitcoin.js parsing?

The only reason why I am asking this is that I would love something that's coin-agnostic, and decoderawtransaction is in all bitcoin clones, but bitcoin.js has to be sometimes specifically ported...

karelbilek commented 7 years ago

...which also applies to block parsing, I guess

dcousens commented 7 years ago

Interesting idea - and it centers around the following lines of code:

https://github.com/bitcoinjs/indexd/blob/master/blockchain.js#L17 https://github.com/bitcoinjs/indexd/blob/master/mempool.js#L36

In terms of cross-coin compatability, those two lines dictate both parsing steps, blockchain and mempool.

I worry about how much extra load you'd be placing on the server by suddenly bouncing thousands of RPC calls. Right now, we only do ~2 per block, retrieving the block height and hex.

dcousens commented 7 years ago

@runn1ng it might be easier to eventually port this to use libconsensus to be honest. Then you could allow alt-coins to swap in "their" version?

karelbilek commented 7 years ago

Hm, I am not 100% sure what exactly is a responsibility of libconsensus, how exactly is the work split between it and bitcoin-core, and how exactly do the altcoins copy these splits. So I can't have a say about that. :(

I know your goals are different, but in my ideal world, there would be one such indexer as indexd, and the indexer could be run on multiple altcoins (some of which diverged from bitcoin significantly). :)

Currently, all the legwork with multiple C++ patchsets (the bitcore patchsets have to be ported to different coins with different codebases), PLUS the different work on bitcore itself, is a PITA.

dcousens commented 7 years ago

@runn1ng it wouldn't be impossible... probably a performance hit. I think if we measure that performance hit... we'd know if it was worth it.

Thoughts? I'm seeing about 25,000 RPC calls/sec on my own node for getrawtransaction, but that is pure hex.

If we use the JSON format (what we'd need to avoid parsing) - it might thrash the UTXO?

This path would also neuter https://github.com/bitcoinjs/indexd/issues/7 as an option.

dcousens commented 7 years ago

@runn1ng would you be up to doing a PR for getrawtransaction, and measure it? It will be slow, but, it might be OK?

dcousens commented 7 years ago

@runn1ng I guess my other question is, how much is time a factor in the importance of this software.

To me, fast sync is great - but, in your case, time is less important compared to the capability to scale to other software.

dcousens commented 7 years ago

Benchmarking this

dcousens commented 7 years ago

Closing in favour of https://github.com/bitcoinjs/indexd/pull/13