sipb / homeworld

Cluster management system for the SIPB Hyades project
MIT License
21 stars 19 forks source link

Use self-contained spire executables instead of administrator packages #494

Open celskeggs opened 4 years ago

celskeggs commented 4 years ago

Currently, we have to install a set of our debian packages in order to administer a system.

However, in theory, we could simply pack all of our dependencies into a single executable, and then not need to install any packages. This would make it easy to administer a cluster from an arbitrary machine.

Key challenges:

The main advantage of this would be that we would no longer need to set up deployment chroots at all, and might even be able to execute spire directly from a bazel build with bazel run //spire. This would decrease development overhead and allow us to eliminate unnecessary code.

However, there are significant trade-offs, so I'm not confident this is the right route forward -- and even if it is, it is not on the critical path.

cryslith commented 4 years ago

We could consider shipping a spire container, which would help with the portability, though not the startup time.

celskeggs commented 4 years ago

Since we already have support for packing arbitrary files into the spire executable through py_binary, I'm not sure what advantage a container would have -- especially since we'd then have to install a container runtime on any machine we want to use for administration.

One solution to the startup time problem would be to have a directory under ~/.cache or somewhere else where we extract all the dependency files, and that we verify against known hashes and replace if different during spire initialization.

That way, we only need to extract a short script from a large zip file, rather than extract the entire file. Though there would be some complications, and we might need to patch py_binary.

(But it would also be worth finding out if startup time is even an issue in practice.)