Add equivalent of FLUSHALL command

[jemc]

Redis has a FLUSHALL command that deletes all keys from all internal keyspaces.

Adding a similar feature to Jylis has been requested by @solnic.

Outstanding questions:

• [ ] How should this behave with disk persistence enabled? Should it also clear the disk files? If not, then the data could get revived from disk in the future.
• [ ] How should this behave when clustered? Should it propagate the deletions to all other instances in the cluster? If not, then I'd expect other cluster instances to eventually revive the old data.

[mfelsche]

If I may add my opinion:

• too keep things easy, if persistence is just replaying the commands, just append the "FLUSHALL". this might be inefficient while replaying, but this can be tackled in a later optimization.
• FLUSHALL should delete all data on all nodes.

[amclain]

In my usage a FLUSHALL command would be helpful for integration tests. However, it's more of a convenience because I am able to achieve the same outcome by running the database without disk persistence and restart it between tests to clear out the memory (using Docker). I'm curious what other use cases could benefit from FLUSHALL.

My concern with the command is the case of a malicious user gaining access to a production database cluster and executing the FLUSHALL command. Maybe that wouldn't be so bad if Jylis is being used as a cache, but definitely not good if it's the primary data store. If FLUSHALL is implemented, maybe it should be opt-in at startup and ignored by cluster nodes that haven't opted in? Basically what I picture is opting a single node in for tests and opting out in production.

[jemc]

@amclain - I have similar concerns. I also hesitate to work on a complicated implementation for a feature that seems to really only be useful in a test environment. Making this kind of operation coherent in a distributed way with respect to CRDTs is nontrivial, and adding that kind of complexity is going to increase defect probabilities.

So, based on both the data security and implementation complexity concerns, I think I want to approach this differently. Rather than deleting data as an operation that affects the disk persistence layer and other nodes in the cluster, I think I want to view this as basically a "restart" command, that would remove all in-memory data structures and actors - basically, act as if you were doing a clean boot of the application, but without having to actually restart the OS process / Docker container.

Basically, this would give a cleaner, faster way to acheive the same effect as the restart-the-docker-container workaround that @amclain and @solnic are already doing.

It wouldn't add any complexity to the data management layer (which resolves the implementation complexity concern) and it seems to resolve the security concerns cited by @amclain because data isn't truly lost if it still exists on disk and/or in other nodes in the cluster - a robust usage of jylis in production that cares about data persistence shouldn't be relying only on the in-memory process anyway.

I'll work on this a bit today.

[amclain]

I think I want to view this as basically a "restart" command

Using SIGHUP or SIGUSR2 may be an easy and secure way to do this. If you implement the command in the protocol just be careful with it being a potential DDoS vector (maybe still opt-in). I'm not sure how much network overhead is created if one node is continuously wiped clean and has to keep resyncing with the cluster -- something to think about.

[jemc]

I've implemented it as SYSTEM FORGETALL: https://github.com/jemc/jylis/commit/c4181b0a624c31095c6847f9fc909a0bab6a51db

Opened #14 to track the remaining security concern.