Caddy 2 integration

[mholt]

I'm very excited to see Smallstep's announcement today, congrats!

You've kind of beat me to the chase -- part of Caddy 2's original design plans was to include an integrated certificate authority for internal key infrastructure. Basically smallstep, but integrated.

I have yet to test it out myself, but from the docs and demos it seems that you're knocking that out of the park already. So I don't see any need to reinvent the wheel here.

What would you like to be added

I want to see smallstep become a Caddy 2 app.

Caddy 2 can run any kind of Go service:

type App interface {
    Start() error
    Stop() error
}

If you can wrap the Smallstep CA server with these two methods and make it configurable via a JSON Unmarshal, then I think we're golden, at least for getting started.

There is a detailed doc here that explains how to write Caddy modules: https://github.com/caddyserver/caddy/wiki/v2:-Writing-a-Module

Why this is needed

Right now, using Smallstep with any web server requires extra steps (no pun intended). This could be eliminated if it "just worked" in the web server directly.

Caddy 2 already nails automatic HTTPS for public domains and certs. It makes sense that it should "just work" in a cluster for internal PKI as well. I think Smallstep can make this possible.

Basically, I envision a simple Caddy config, something like this:

{
    "apps": {
        "smallstep": {
            // smallstep config goes here
        },
        "http": {
            // reverse proxy config goes here
        }
    }
}

One possible advantage here is that the ACME/CA server could potentially run in a distributed fashion similar to how Caddy does for certificate management: by configuring the fleet to use the same storage, it can automatically coordinate cert management. I see an advantage here for Smallstep doing something similar for its issuance processes.

Now, maybe this exact way of doing the integration isn't optimal. But we should talk about it. I want to open the discussion to see how we can integrate Caddy 2 and Smallstep. I can help with the integration, but I can't do it all myself.

Also, I'd be happy to invite your team to our development Slack. Just let me know which emails to invite.

[maraino]

Caddy's http and tls are apps in Caddy 2: https://github.com/caddyserver/caddy/blob/v2/modules/caddyhttp/caddyhttp.go https://github.com/caddyserver/caddy/blob/v2/modules/caddytls/tls.go

[mholt]

@maraino And this is a good point, maybe a better place to integrate would be within one of the existing apps (probably the tls app) -- definitely up for discussion.

What are some steps to the process that you wish you could eliminate to make this even better? Let's leverage Caddy's strengths with smallstep's and see what is possible.

[maraino]

@mholt I was just adding those links as a reference. I've been looking at those files, and I think the integration would be quite straightforward.

Without taking into account how the password are managed, a simple app would look like:

type App struct {
    *authority.Config
    srv *ca.CA
}

func (a *App) Start() error {
    srv, err := ca.New(a.Config)
    if err != nil {
        return err
    }
    go srv.Run()
    a.srv = srv
    return nil
}

func (a *App) Stop() error {
     return a.srv.Stop()     
}

Things to take into account:

1. How to obtain the passwords? config?, environment variables?
2. Is there any way to support reloads?
3. Instead of using ca.Run() add a new method that receives a net.Listener so we can check that we are able to listen in that port and return an error.
4. ...

[mholt]

That looks like a good start for an app, definitely.

How to obtain the passwords? config?, environment variables?

Sure, either of those is fine -- note that plaintext passwords in the config means that -- in an automated environment -- it's probably stored in plaintext elsewhere, so if the configs are secure, then that shouldn't be a problem for most threat models. Caddy's configs are ephemeral (i.e. there's not necessarily any config files, so config can be POSTed over a secure socket and then is stored in memory only, generally) so storing passwords in them isn't the worst thing.

Env variables are also a possibility of course. In Caddy we prefer to keep the config as consolidated as possible, though: i.e. all together in one place, rather than relying on system environment, CLI flags, or anything outside the JSON config. So I think my vote would be for the JSON document itself. (Maybe wait for a feature request before adding env variables?)

Is there any way to support reloads?

Caddy can manage listeners gracefully through reloads (caddy.Listen), but the specifics of reloads will depend on your app. We can look at this together if you'd like some guidance. (I am not yet familiar with smallstep's internals but I suppose that will change.) These graceful reloads also work on Windows!

Instead of using ca.Run() add a new method that receives a net.Listener so we can check that we are able to listen in that port and return an error.

Yep, ideally, you'd have a way to be passed a listener that is already listening on a port. (You'd get it from caddy.Listen.)

[mholt]

This is still a WIP, but as an update:

• Caddy can issue certificates with proper PKI, including fully-managed CA certs
• Caddy can embed a Smallstep ACME server as of tonight (https://twitter.com/mholt6/status/1243408326311854080)

Still lots to do, but the big pieces are starting to come together.

Next steps, probably:

• Polish current features a bit (will need some help on your end for a few little things, I will open issues about those as needed)
• Enable using ACME to get client certificates (not really Smallstep-specific, but useful for a lot of Smallstep users)

[sourishkrout]

Feel free to spin out new tickets. Happy to chat on Slack if there are any questions/plans. Awesome integration in Caddy2! Check out https://caddyserver.com/docs/automatic-https