Open kevincox opened 8 years ago
I believe @FPtje worked on an automatic rollback system which is being used in production by @basvandijk at LumiGuide, as discussed in this great presentation at NixCon: https://youtu.be/J4DgATIjx9E?list=PLgknCdxP89ReQzhfKwMYjLdwWsc7us8ns&t=1233
As Bas mentioned in the talk I think this would be a fantastic as a NixOS package, though I'm unsure if it's being worked on by anyone at the moment.
It would be super snazzy if nixops had an automatic rollback system so that bad configs would cause the machine to rollback automatically rather then possibly becoming "bricked".
I imagine it would be something like a series of checks on a timer after every deploy. So if 60s after deploy the checks don't all pass it would trigger a rollback automatically.
A nice default check to have would be SSH access. This is because if you can't get SSH access you can't deploy new configs. I don't know the best way to implement this but a simple implementation would be a new SSH connection after deploy that would try to make a file and then the checks assert that the file exists. Or it would also be possible to not disconnect from the nodes until the checks finish. So do the SSH check while still logged in and if it fails use the existing deploy connection to rollback the machines.