search

owenkellogg / ripple-historical-database

SQL database as a canonical source of historical data in Ripple
3 stars 2 forks source link

Initial importer #3

Open sublimator opened 10 years ago

sublimator commented 10 years ago

I just looked @ ripple-ledger-downloader, and noted that it uses the ledger command, with expand:full and transactions:true to get historical data.

It would be relatively trivial to create a command line utility using ripple-lib-java to import data from transaction.db / ledger.db. Both compress down to around a 12gb .xz these days, so you can play with them locally. Not sure if that's of any use, but it's potentially faster (bulk requests, core utilization)

Similarly, I'm working on doing dumps of account state trees, as a sequence of leaves, for every $n ledgers and deltas (deleted/modded/added leaves). This is also potentially faster than rippled (in some configurations) and perhaps useful if you are going to be creating statistics involving the whole ledger (or say ledger item tables keyed by ledger_index)

owenkellogg commented 10 years ago

The problem with doing sql dumps from a rippled server is that the system would not be able to be reconstructed by an outside party using only the rpc interface.

sublimator commented 10 years ago

Do you ever see history servers getting history from other history servers or they will always get it from rippled?