search

mholt / caddy-events-exec

Run commands on Caddy events
33 stars 2 forks source link

Roadmap for integrating this in Caddy? #8

Closed vmsh0 closed 6 days ago

vmsh0 commented 1 month ago

Hi, Many thanks for this useful Caddy plugin! Is there a roadmap for integrating this in vanilla Caddy?

Currently, this is the only event handler that is mentioned in the docs: https://caddyserver.com/docs/caddyfile/options#event-options

This functionality would be useful to me; however, not at the price of compiling my own Caddy instead of using the one provided by either my Linux distribution or from your repositories.

mholt commented 1 month ago

Not likely, at least not in the near-term. The module would need to see waaaaaay more widespread use and vetting in the wild, because it has the potential to run arbitrary commands.

So for now, it will stay separate, until we can ensure it is safe from all angles. (I think it is safe though, I can't possibly imagine a scenario where you can get RCE or ACE, but I'm not an expert in that either.)

Hopefully that makes sense. If enough people use it, we can consider it. If not, there's no information to go on.

vmsh0 commented 1 month ago

Thank you for your responsiveness, it is much appreciated. Do you think there is any alternative to react to new certificates and deploy them to other software, other than polling with a cron job (or recompiling Caddy with this)?

mholt commented 1 month ago

What is the "price" you are talking about exactly, with regards to building Caddy with this plugin? I guarantee Caddy's certificate management is an order of magnitude more effective/robust than any cron-based solution and comes with a much lower "cost" than other solutions.

vmsh0 commented 1 month ago

Caddy is great. I agree that it's an incredibly solid piece of software. Caddy's certificate management is much more reliable than anything else I could ever implement. So there's no question of using it or not. The only matter under discussion is picking among the two following options:

To me, this is a no brainer: the first option is unsupported and still includes essentially all of the possible failure modes of the second one. The elegance of the event-driven approach is immediately eliminated by the fact that we are replacing the deployment of a reputable package with world-class quality control, with me compiling stuff when a new patch comes out.

In other words, the cost of the first option is much higher, and it also has long-term ripercussions. I also do not see how the cron (or systemd) solution would be an order of magnitude less effective or robust. I do not say this as an exercise: I actually really do not see it, and I am geniuinely interested in hearing what you think the limitations would be.

mholt commented 1 month ago

Hmm, thanks for the explanation. Let me explore things a little more.

To me, this is a no brainer: the first option is unsupported and still includes essentially all of the possible failure modes of the second one.

What do you mean by unsupported? I assume you're referring to this:

pulling this plugin and rebuilding network-facing Caddy every time a new security patch comes out

We have tools like xcaddy and our Download page to do the builds for you. (Just please don't automate the download page, unless it's very politely done... and don't expect an SLA. It's intended for manual use really.)

Also of note, the last material update to this plugin was 2 years ago; and Caddy has releases only once every few months... a couple times a year on longer dev cycles. So "every time" is not as frequently as you might think. :slightly_smiling_face:

And what is really the difference between:

pulling this plugin and rebuilding network-facing Caddy every time a new security patch comes out

and (my words):

pulling Caddy without plugins and rebuilding network-facing Caddy every time a new security patch comes out

Caddy with plugins and Caddy without plugins is virtually the same using xcaddy or the Download page.

me compiling stuff when a new patch comes out.

This is arguably safer than downloading prebuilt binaries.

I also do not see how the cron (or systemd) solution would be an order of magnitude less effective or robust. I do not say this as an exercise: I actually really do not see it, and I am geniuinely interested in hearing what you think the limitations would be.

Cert management is a complex, involved, long-running process, that only the process itself can really understand. What I said is true for the same reason that tools like Certbot or lego should only be used to help transition legacy sites to HTTPS. (Even the lead engineer of Certbot recently asked if they should just recommend Caddy.) Having the cert management centralized as a core part of a system is the industry recommended best practice to improve reliability and decrease moving parts.

I understand that statically-compiled servers may be a new concept that some sysadmins are uneasy about shipping, but in reality, static binaries are more reliable and simpler in production than managing dynamically linked environments or similar where you have a lot of moving parts that don't really understand each other.

vmsh0 commented 1 month ago

Thank you for your roundup. I appreciate the time you're putting into this conversation!

I still disagree, I think the big contention point is that pulling from Caddy and rebuilding is in your opinion safer than pre-built binaries. I think that is just not true when you get your binaries from a reputable source.

On that matter, I don't understand how your download page is any better than my distro: if anything, it's much worse on the fronts of not having a multi-decade excellent track record and not supporting automation.

I will not elaborate further, as this is a very old discussion that I'm sure has played out countless time with the same results and conclusions. I just wish to share this (doi.org/10.1145/358198.358210) very famous article by Ken Thompson; it's from 1984 but I think it's still very relevant to this discussion. What I have myself gathered is that the only really relevant thing in computer security is the human chain of trust. This is why I really like my distro's binary packages, and not so much downloading your source code and compiling it with your tool. I trust my distro much more than the Caddy project, plain and simple :bowtie:

I also disagree with this statement:

So "every time" is not as frequently as you might think.

This is a non-argument, because security vulnerabilities require immediate fixes regardless of the usual release cycle length. When that happens, I'm usually ok with dropping everything for 10 minutes to upgrade a package outside my usual maintenance window -- I would be less thrilled to have to recompile software, no matter how much nicer the experience is when using custom tools such as xcaddy.

nathanvy commented 6 days ago

I'll wade into this discussion uninvited:

The xcaddy thing is currently dissuading me from using this plugin. I want to use Caddy for personal projects, and I just want to be able to issue an apt update && apt upgrade. If I do that, though, my understanding is that apt will clobber my custom xcaddy build.

Therefore it seems my options are as follows:

  1. Use cron or similar to just pull the certs unconditionally, eschewing the Caddy events system entirely
  2. Maintain a custom build of Caddy that will get clobbered every time I upgrade my system packages, which means additional manual steps after apt completes, steps that I might forget, which sort of defeats the purpose of package management
  3. Don't install caddy from the apt repos at all and just build from source, which again defeats the purpose of package management
  4. Go back to certbot+nginx

Frankly none of those seems appealing, but it seems to me that, for users like me who want to be able to use domain certs for additional services that aren't serving HTTP, the certbot+nginx combo seems superior to Caddy.

Just my 2 cents.

vmsh0 commented 6 days ago

FWIW, I have reached the slightly different conclusion that I'm just going to poll the Caddy certificates directory once a day. It might break at some point because of changing directories, but that's a much more trivial fix with respect to fixing something that doesn't compile anymore (and has much less severe security implications too, not requiring to steer away from the software chain of trust of my distribution).

mholt commented 6 days ago

The xcaddy thing is currently dissuading me from using this plugin. I want to use Caddy for personal projects, and I just want to be able to issue an apt update && apt upgrade. If I do that, though, my understanding is that apt will clobber my custom xcaddy build.

Honestly, that's a great complaint to take up with apt for not supporting statically linked custom builds.

Anyway, this is getting off-topic, and I think I've answered the initial question, so I'm closing now.