valeriansaliou / sonic

🦔 Fast, lightweight & schema-less search backend. An alternative to Elasticsearch that runs on a few MBs of RAM.
https://crates.io/crates/sonic-server
Mozilla Public License 2.0
20.1k stars 578 forks source link

Configure Backup and Restore default path for Startup and Shutdown #169

Open StevenACoffman opened 5 years ago

StevenACoffman commented 5 years ago

Thank you for a very, very fast and lightweight tool! It's already proven it's value.

5 looks like it added a control channel option to backup and restore the KV and FST

I would like a configuration option to default the backup/restore path, and if it is set and backup files are found there on startup, to restore it without issuing a trigger command, and on shutdown to backup to that location without issuing a trigger.

Periodically, I would like to trigger a backup, but not need the client to be aware of the path that was configured.

What do you think?

StevenACoffman commented 5 years ago

By the way, when I TRIGGER backup /var/lib/sonic/store/a via telnet, I generally get an ERR internal_error (once I got an OK), but I never seem to get anything but empty directories written out. Not sure if I need to perform other commands first (consolidate, flushc?).

valeriansaliou commented 5 years ago

Hello! Thanks!

I accept PRs on this; you can modify Sonic's code yourself and improve the protocol and backup / restore commands for your needs and submit them there.

Regarding the ERR vs OK, is Sonic providing any debug log when the ERR occurs?

alexisvisco commented 5 years ago

@valeriansaliou I would love working on this, I have been doing some rust in the past, maybe I will need some help of you doing a code review after I will submit the PR haha :)

I will open a WIP pull request in some days.

valeriansaliou commented 5 years ago

@alexisvisco Hello Alexis! That would be perfect. As far as I understand, if a default path is configured, then we can allow passing a relative path that will get transformed to an absolute path internally.

As far as security is concerned, we'd probably be better forbid the use of absolute paths and rely on relative paths only, using the configured chrooted backup folder.