Wish: New derivation that uses checkpointing for faster start-up

[bennyandresen]

clj-nix is a great project and my preferred way to build anything clj-related (uberjars or native-images, mostly.) (So thanks for that!)

Please see this blog post: https://yizhepku.github.io/clojure-crac/

The above proof-of-concept could serve as an idea for another way to improve the customJdk derivation.

[jlesquembre]

I didn't know about the CRaC project, it looks quite interesting, thanks!

It makes sense to add another helper function, similar to customJdk, to create a derivation with CRaC. But unfortunately, we need a custom jdk build: https://github.com/CRaC/openjdk-builds Which is not on nixpkgs yet.

Also (and correct me if I'm worng), looks like if jdk-crac is needed, we can't use jlink to reduce the jdk size. In that case, we need to choose between jlink to reduce the derivation size, or CRaC to reduce the startup time.

To summarize, I like the idea, but I'm going to wait until CRaC is more mature.

But feel free to start to explore that space. We could add a new function, and mark it as experimental. Happy to get a PR about it.

[bendlas]

A while back, I've played around with checkpointing a clojure runtime with just CRIU. It seemed to work, even without special runtime support.

This worked basically like: Set up -> print "ready" to stdout -> wait for "go" + arguments on stdin -> run main

A few provisions were necessary, also documented with CRIU:

• the linux kernel module (obviously)
• run in a new process namespace, so that duplicate PIDs are allowed when restoring processes in parallel
• don't do weird stuff during setup; usually just load namespaces, warm up jit, gc
  • open files are linked back up, I believe, but must be available
  • sockets and file descriptors: probably not

As I said, it seemed to work, but I didn't test it deeply. That said, CRaC doesn't seem to do much more either, than to tell you to not do funky stuff with FDs before restore: https://github.com/CRaC/docs/blob/master/fd-policies.md

EDIT check it out: https://github.com/webnf/lein-nix/blob/master/lein-criu.nix

• an obvious problem with that experiment is that it uses a custom setuid binary, to run sudo criu during build, for checkpoint generation. This could be done as a separate step though
• I don't think it has the PID namespace yet, so you'd have to use unshare, to start multiple restores in parallel
• ~it's still based on leiningen ;-)~ actually, the experiment was to get a faster-starting leiningen, therefore it does part of the preload sequence in a leiningen task.