search

bitcoinjs / indexd

An external bitcoind index management service module
ISC License
53 stars 23 forks source link

Use node parsing instead of bitcoinjs #13

Closed dcousens closed 6 years ago

dcousens commented 7 years ago

Speed is about the same as I didn't know getblock had a verbosity argument to include decoded transactions. (See https://github.com/bitcoin-dot-org/bitcoin.org/pull/1769)

Database breaking change (https://github.com/bitcoinjs/indexd/pull/13/commits/87d9bc3ab103111c0edd86bec91b3f93c6b28cf3) as I changed types.tip type to include height. Easily migrated if anyone needs to ask how.

karelbilek commented 7 years ago

Oh wow.

I am now super busy, but I will check speed when I have free time.

dcousens commented 7 years ago

Genesis to 481864 estimated at around 22 hours... not amazing, but, usable.

Using fast-dat-parser, you can expedite that down to around 30-40 minutes... but you would forfeit the 2nd order data (feeindex), simply as I haven't implemented that index in fast-dat-parser.

dcousens commented 7 years ago

Time taken was 40 hours

karelbilek commented 7 years ago

The 30-40 minutes with C++ parser is including the leveldb database operations?

just asking, since bitcore takes 2 days to synch, but it is disk-bound on the db operations. So this seems unbelievably fast to me; but maybe it is better optimized :)

dcousens commented 7 years ago

@runn1ng the indexes are insanely trivial. 40 minutes was what I remember my last experience being.

I'll time it when I do it again soon :+1: .

dcousens commented 7 years ago

I'll run a full-parse soon as part of https://github.com/bitcoinjs/bitcoinjs-lib/issues/881#issuecomment-327139428, and let you know

dcousens commented 6 years ago

Still have this on my TODO to benchmark fast-dat-parser as a comparison