search

stellar-deprecated / stellard

INACTIVE. Server in the Stellar network. Maintains the distributed ledger. Introduces and validates transactions. This repo is not in active development, it is being replaced by stellar-core.
Other
271 stars 60 forks source link

Bulk replay #151

Closed graydon closed 9 years ago

graydon commented 9 years ago

This adds bulk ledger-export and replay to stellard. There are three new command line flags (two for bulk operations and one diagnostic one that just shows a ledger to stdout). It appears able to export the first million ledgers, which I have in my workstation's nodestore; re-importing them runs out of memory somewhere in the high 300,000s in the current codebase; I'll continue to work on tuning that with @MonsieurNicolas but this looks like a good time to merge this work in.

MonsieurNicolas commented 9 years ago

You were talking about a special flag/or range check to, I think, not use the actual transactors but use the meta info to move forward - this was to avoid having to deal with bug fixes that changed the behavior of certain operations. If you are going to introduce such logic, I am not sure we'd want to revert any of the changes in the code (and just snap to recent ledgers that are running with whatever the code does today).

graydon commented 9 years ago

Updated with "historical quirks" table, that enables/disables behavior that's been fixed in the past, during replay only. Should pass tests now. The ledger number ranges for bounding the quirks are not quite right yet (I will hunt down precise bounds next week) but they're good enough to get 100k of replay in and the tests pass, so it's just a matter of tightening the numbers up.

graydon commented 9 years ago

Updated to fix accidental commented-out same-currency quirk, tightened quirk window on lsfPasswordSpent to get replay back up into the "workstation runs out of memory" range. Will see how far it gets on a larger AWS machine later.

graydon commented 9 years ago

Updated to fix two bugs preventing offset-replay from working (and failing to actually write permanent SQL files on reload).