jechols
commented
2 months ago Leaning toward a daemon, as it'll be easier to secure a separate process that doesn't offer any shell access, easier to debug when things go wrong, and likely more stable in the long-run if we need this to live for a while.
Also leaning toward the lower-level Glider Labs ssh package. It still wraps the well-tested Go package, like wish does, but it has significantly fewer dependencies, and does everything we'd need.
Looking at a Go ssh blog ("Executing commands via SSH using Go"), the actual NCA side of this should be pretty basic.
Go's got an ssh package with some pretty solid support for building ssh clients and servers. We could leverage this to build a job that starts an ssh session into staging/prod and runs a batch load / purge command.
For baremetal this is pretty trivial: configure the ssh server names somewhere in NCA, and go. For a containerized setup, or local dev, it seems like it could be tricky, but it's really just a matter of having a dedicated "command" server that exposes ssh (internally, not to the world!). It'd be a bit like situations where production is under heavy enough load that batch processing is done on a separate system that connects to the same Solr and MariaDB servers.
Alternatively, we could have a daemon running on staging/production that listens for load/purge requests and runs them. This would make migration to a real Django API a bit easier potentially. But I think it's also more work up front, so I'm not sure I'll put the time into this approach. A bit more research into something like Charm's "wish" might be worthwhile, though. If we never get Django into a good state for APIs (or it turns out to just not handle them well even when Django 5 drops), it wouldn't hurt to have a small, focused daemon that's solely responsible for simple SSH command execution.
Note:
golang.org/x/crypto/sshis under the "x" sub-repository. Some people think means "experimental", but it actually means "external", as in not part of the core Go standard library. These packages generally have full supported, but on rare occasions get deprecated, and sometimes have breaking changes applied. In other words, it's still a core and well-supported package, it's still more likely to stick around than a random non-Go project, it just doesn't have quite the same guarantees as the standard library.I write this not because I worry somebody would challenge me, but because... I just had to look it up since I wasn't sure I wanted to add an experimental package. Yes, I am in fact "some people" in this scenario.