search

sigp / lighthouse

Ethereum consensus client in Rust
https://lighthouse.sigmaprime.io/
Apache License 2.0
2.87k stars 724 forks source link

Add `lighthouse db purge dht` subcommand #2824

Open pawanjay176 opened 2 years ago

pawanjay176 commented 2 years ago

Description

Currently, we persist the lighthouse dht on exit and load from the db on startup. The persisted dht is stored in the hot db under the DhtEnrs column.

This makes it inconvenient to start from a fresh dht as we would have to delete the specific column from the hot db on startup. Also it seems a bit weird to persist the networking dht in the beacon chain hot db as you don't need the dht to load the chain.

IMO, it would make more sense to store it in a separate file under the beacon/network directory. This would allow us to simply delete the dht file if we want to start from a fresh dht. Deleting the entire beacon/network directory would also give us a fresh networking node where we get a new network identity and just the bootnodes from the provided config (equivalent to --purge-db for the beacon chain).

michaelsproul commented 2 years ago

I think we should be careful about adding any more pseudo-databases due to the trouble we've had in the past with atomicity and crash safety. I think a separate Lighthouse command to delete the DHT could be good, like lighthouse db delete-dht (rather than a fiddly flag like --purge-db).

Alternatively if you want a separate file in the network dir you could use a separate DB, either LevelDB or SQLite. That would outsource the file I/O concerns to the database

pawanjay176 commented 2 years ago

I was suggesting writing to a file since it's not a big deal even if it gets corrupted; we can start from a fresh dht with just the bootnodes and get a good peer count pretty fast.

However, I like your lighthouse db delete-dht idea better. Is adding a subcommand for db operations something that would be useful for stuff other than deleting the dht as well e.g. manual migrations of some sort?

michaelsproul commented 2 years ago

Yeah I've been thinking of adding a db sub-command for a while. We could have lighthouse db info for stats, lighthouse db purge, a command to drop the op pool, migrations, etc

michaelsproul commented 2 years ago

There's some WIP code here that we could adapt: https://github.com/sigp/lighthouse/pull/2685