caddy: add all addons

[davidak]

compile caddy with all addons listed on the download page.

Only middleware handlers which are invoked from the Caddyfile will be chained in, so small Caddyfiles are very fast and efficient.

source: Directive section on https://caddyserver.com/docs/caddyfile

a later improvement could be to select just the addons you want like vim_configurable.

@angus-g

[angus-g]

This does sound like a good idea to improve the usefulness of caddy, but it's quite non-trivial from what I can see. It looks like there's some code generation required (in https://github.com/caddyserver/caddydev/blob/master/caddybuild/build.go) to insert the custom directives into caddy.

If that's the case, it would probably be easier to start with desired addons and do the code generation at build time. I expect this would need a bit more Nix code than currently exists for the package in order to support hashes for caddy and all addons, then modifying and building the source after this. I'm not great with the language just yet, so I'm not even sure of the feasibility of this. (Anything's possible, right?)

[davidak]

I found a Dockerfile where someone installed caddy with the git addon. That is also the one i want to use.

https://github.com/jumanjihouse/docker-caddy/blob/master/Dockerfile.build#L34

You could write the nix-expression like for a general program, but the go infrastructure is different. You should ask in the IRC or Mailinglist how to realize that best. I'm also not that experienced with nix, never packaged a go program.

[davidak]

you can override build phases like described here https://nixos.org/nixpkgs/manual/#ssec-controlling-phases

for example the preBuildPhases like here: https://github.com/NixOS/nixpkgs/blob/dfc6d42860b41042deeccb487c73adb138a36e48/pkgs/top-level/go-packages.nix#L642

then you can use caddyext install git in the build phase.

if you are experienced with go, you can do that. i could also figure it out and create a PR. how you like :)

[davidak]

we can also create an additional package of caddyext and use it as caddy buildInputs.

but i see 2 problems:

• the hash of the addon source is not checked
• reproducible builds are impossible since the source of the addons change over time

[0xABAB]

If the source of the addons changes over time, I would try to convince upstream to change their ways and otherwise completely ignore this project, for a lack of professional software development practices.

Sometimes "gratis" software is even too costly.

[davidak]

@0xABAB you may misunderstood me. caddy has it's own "package manager" for addons.

if i execute caddyext install git now and again in one year, i will get a different version. if we build our nix package this way, it will not be reproducible.

i asked the developer of caddyext if it can install from local directory. then we can clone the addons repository and checkout a specific version.

Caddy is free software.

[davidak]

in version 0.9 addons will be plugins. i havn't looked what that means for building from source.

https://forum.caddyserver.com/t/caddy-0-9-beta-version-available/146?u=matt

we should wait for 0.9 to integrate the fix for this issue into that.

[joepie91]

Any updates on this?

[bricewge]

It is now quite easy to add plugins in caddy: https://github.com/mholt/caddy/wiki/Plugging-in-Plugins-Yourself But I don't know how to make the build reproducible. It seems that at most the major version can be specified , not a specific version or a commit. For example adding _ "gopkg.in/restic/caddy.v0" will for the moment use the v0.2 as explained here.

[bricewge]

I managed to add the restic plugin to caddy by adding thoses lines to the current derivation:

    preConfigure = ''
        substituteInPlace ./caddy/caddymain/run.go \
        --replace '// This is where other plugins get plugged in (imported)' \
        '_ "github.com/restic/caddy"'
    '';
    goDeps = ./deps.nix;

And adding deps.nix:

[
  {
    goPackagePath = "github.com/restic/caddy";
    fetch = {
      type = "git";
      url = "https://github.com/restic/caddy";
      rev = "v0.2.0";
      sha256 = "0wjhbnm405vdpf3jwi9dzhz6nd5rhmxqibsa1nx2iyq43bc3p6sk";
    };
  }
  {
    goPackagePath = "github.com/restic/rest-server";
    fetch = {
      type = "git";
      url = "https://github.com/restic/rest-server";
      rev = "v0.9.3";
      sha256 = "0f6i952iy4arnc96wpayvh9wa1mdw7vprwwbnjxwj09jvifqd0hp";
    };
  }
]

So it is now really easy to add some plugins to caddy! Whoever I don't know in which form this configuration should be available in NixOS? Maybe having two versions would be the easiest:

• caddy as it currently is, without any plugins
• caddy-full with all the plugins availaible

[davidak]

I think that 2 versions would be OK.

Another option would be to enable each plugin like described for Chromium and Firefox here https://nixos.wiki/wiki/Chromium

[JakeStanger]

The above method for me isn't working. Has anybody had any luck or got a better approach?

[CMCDragonkai]

According to the https://github.com/caddyserver/caddy/wiki/Plugging-in-Plugins-Yourself

The current derivation is at: https://github.com/NixOS/nixpkgs/blob/master/pkgs/servers/caddy/default.nix

All we need to do is is something like:

{ caddy }:

caddy.overrideAttrs (attrs: {
  preBuild = ''
    cat << EOF > caddy/main.go
    package main
    import "github.com/caddyserver/caddy/caddy/caddymain"
    import _ "github.com/techknowlogick/caddy-s3browser"
    func main() {
      caddymain.EnableTelemetry = false
      caddymain.Run()
    }
    EOF
  '';
})

Which I override the preBuild to write our own main.go to be used.

You can see that I added in:

    import _ "github.com/techknowlogick/caddy-s3browser"

However that dependency is not being brought into the buildGoModule.

Normally with buildGoPackage, I'd add extra things to deps.nix. However here, this is buildGoModule, so how do we add caddy-s3browser as an extra dependency to buildGoModule?

[CMCDragonkai]

There's a overrideModAttrs in the buildGoPackage function. However I can't seem to access that in the resulting derivation. As in pkgs.caddy.override cannot access overrideModAttrs and neither can pkgs.caddy.overrideAttrs.

Instead it seems like it is needed to override the go-modules derivation located in pkgs.caddy.go-modules.

No documentation on how to override the go-modules derivation to add an extra go package into the go-modules.

[CMCDragonkai]

I think what needs to be done is that the go-modules.buildPhase needs to be overridden. Like a preBuild hook.

Because that's when go mod download is called.

Either that or the go.mod must have a patch applied to add that in prior to the source being used. I'm wondering if go.sum also needs a patch.

This does not work even if I update go.mod and go.sum. I think this is not working for overriding caddy. It seems some thing more drastic required.

The buildGoModule builder needs to have a better way of adding extra go dependencies.

[CMCDragonkai]

Abandoned trying to do overrides. I just had to create my own Go package to do this. But in the future, caddy needs some work to be able to compile plugins more easily like how firefox/chrome does its plugins.

[ooesili]

@CMCDragonkai Hey there! I couldn't get overrides to work either, but I ended up with this and thought I'd share. I ran into some issues with nix caching old versions of the module after tweaking the derivation but leaving modSha256 the same as before. Nix thought it had already built those modules even though the inputs technically changed. It tripped me up for a while, maybe it happened to you too? Anyways here's my local fork of caddy:

with pkgs; buildGoModule rec {
  pname = "caddy";
  version = "1.0.3";

  goPackagePath = "github.com/caddyserver/caddy";

  subPackages = [ "caddy" ];

  src = fetchFromGitHub {
    owner = "caddyserver";
    repo = pname;
    rev = "v${version}";
    sha256 = "1n7i9w4vva5x5wry7gzkyfylk39x40ykv7ypf1ca3zbbk7w5x6mw";
  };
  modSha256 = "1xs5qf89sddj21vig4lmhdfkyccf2alrd39915spz845xhv3pl8w";

  overrideModAttrs = (old: {
    preBuild = ''
      go mod edit -require=github.com/tarent/loginsrv@v1.3.1
      go mod edit -require=github.com/BTBurke/caddy-jwt@v3.7.1+incompatible
    '';
  });

  preBuild = ''
      cat << EOF > caddy/main.go
      package main

      import (
        "github.com/caddyserver/caddy/caddy/caddymain"

        _ "github.com/BTBurke/caddy-jwt"
        _ "github.com/tarent/loginsrv/caddy"
      )

      func main() {
        caddymain.EnableTelemetry = false
        caddymain.Run()
      }
      EOF
  '';

  meta = with stdenv.lib; {
    homepage = https://caddyserver.com;
    description = "Fast, cross-platform HTTP/2 web server with automatic HTTPS";
    license = licenses.asl20;
    maintainers = with maintainers; [ rushmorem fpletz zimbatm ];
  };
}

[stale[bot]]

Hello, I'm a bot and I thank you in the name of the community for opening this issue.

To help our human contributors focus on the most-relevant reports, I check up on old issues to see if they're still relevant. This issue has had no activity for 180 days, and so I marked it as stale, but you can rest assured it will never be closed by a non-human.

The community would appreciate your effort in checking if the issue is still valid. If it isn't, please close it.

If the issue persists, and you'd like to remove the stale label, you simply need to leave a comment. Your comment can be as simple as "still important to me". If you'd like it to get more attention, you can ask for help by searching for maintainers and people that previously touched related code and @ mention them in a comment. You can use Git blame or GitHub's web interface on the relevant files to find them.

Lastly, you can always ask for help at our Discourse Forum or at #nixos' IRC channel.

[davidak]

this is still important

[stale[bot]]

I marked this as stale due to inactivity. → More info

[Garionion]

this is still important

[ljani]

Here's my take based on https://github.com/NixOS/nixpkgs/issues/86349#issuecomment-624489806:

overlays/caddy/default.nix:

final: prev: {
  caddy = (prev.caddy.override {
    buildGoModule = args: prev.buildGoModule (args // {
      vendorSha256 = "sha256-445MYvi487ls6g6i30UTTK2/n2wbILgJEuwNUQE//ZE";
      patches = [ ./0001-plugins.patch ];
    });
  });
}

overlays/caddy/0001-plugins.patch:

diff --git a/cmd/caddy/main.go b/cmd/caddy/main.go
index 2383546..59e467e 100644
--- a/cmd/caddy/main.go
+++ b/cmd/caddy/main.go
@@ -31,6 +31,7 @@ import (

        // plug in Caddy modules here
        _ "github.com/caddyserver/caddy/v2/modules/standard"
+       _ "github.com/caddy-dns/duckdns"
 )

 func main() {
diff --git a/go.mod b/go.mod
index 6fe6902..68bfedd 100644
--- a/go.mod
+++ b/go.mod
@@ -6,6 +6,7 @@ require (
        github.com/Masterminds/sprig/v3 v3.1.0
        github.com/alecthomas/chroma v0.8.2
        github.com/aryann/difflib v0.0.0-20170710044230-e206f873d14a
+       github.com/caddy-dns/duckdns v0.3.1
        github.com/caddyserver/certmagic v0.13.1
        github.com/dustin/go-humanize v1.0.1-0.20200219035652-afde56e7acac
        github.com/go-chi/chi v4.1.2+incompatible

Not exactly pretty, but at least it works:

> /nix/store/aly0qbnxh2drph0gmbbmrcrrijc2m809-caddy-2.4.1/bin/caddy build-info | grep duckdns
github.com/caddy-dns/duckdns v0.3.1
github.com/libdns/duckdns v0.1.1

[curbengh]

I find https://github.com/NixOS/nixpkgs/issues/89268#issuecomment-636529668 works better than previous comment's because it is more caddy-version-agnostic.

[phaer]

Just in case, somebody else who isn't too familiar with nix golang infrastructure is trying to build a current Caddy with plugins: I've updated the package file by @diamondburned in #89268 (comment). Uses fetchFromGitHub instead of fetchGit and declares the current maintainer. I also needed to set "runVend = true" to get all plugin dependencies

pkgs/caddy.nix

```nix { lib, fetchFromGitHub, buildGoModule, plugins ? [], vendorSha256 ? "" }: with lib; let imports = flip concatMapStrings plugins (pkg: "\t\t\t_ \"${pkg}\"\n"); main = '' package main import ( caddycmd "github.com/caddyserver/caddy/v2/cmd" _ "github.com/caddyserver/caddy/v2/modules/standard" ${imports} ) func main() { caddycmd.Main() } ''; in buildGoModule rec { pname = "caddy"; version = "2.4.6"; runVend = true; subPackages = [ "cmd/caddy" ]; src = fetchFromGitHub { owner = "caddyserver"; repo = "caddy"; rev = "v${version}"; sha256 = "sha256-xNCxzoNpXkj8WF9+kYJfO18ux8/OhxygkGjA49+Q4vY="; }; inherit vendorSha256; overrideModAttrs = (_: { preBuild = "echo '${main}' > cmd/caddy/main.go"; postInstall = "cp go.sum go.mod $out/ && ls $out/"; }); postPatch = '' echo '${main}' > cmd/caddy/main.go cat cmd/caddy/main.go ''; postConfigure = '' cp vendor/go.sum ./ cp vendor/go.mod ./ ''; meta = { homepage = https://caddyserver.com; description = "Fast, cross-platform HTTP/2 web server with automatic HTTPS"; license = licenses.asl20; maintainers = with maintainers; [ Br1ght0ne ]; }; } ```

example.nix

```nix pkgs.callPackage ./pkgs/caddy.nix { plugins = [ "github.com/mholt/caddy-webdav" "github.com/greenpau/caddy-git" "github.com/greenpau/caddy-auth-portal" "github.com/greenpau/caddy-authorize" ]; vendorSha256 = "sha256-K2i9MlrR8PMDHHW7V+d5mRgtgb0H5okMz/TlEncvgIo="; } ```

[shadowrylander]

@phaer So should I add this to my list of overlays? Also, how would I go about getting the vendorSha256, if I really need it?

[phaer]

@shadowrylander Yes. Just set an empty vendor hash and nix will fail with the correct one in an error message

[shadowrylander]

@phaer Got it; thanks!

[nixos-discourse]

This issue has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/unable-to-build-caddy-with-custom-modules/19125/1

[jdheyburn]

If anyone was getting some errors when building after updating their nixos-unstable commit, the fix for me was to replace buildGoModule with buildGo117Module - helpfully highlighted in this post.

[winterqt]

@jdheyburn How was it failing with Go 1.18? It builds fine on Hydra, and plugins that work on 1.17 should be compatible with 1.18, in most cases.

[tjni]

While I do not think it meets nixpkgs requirements for reproducibility[^1], it was enough for my use case to build a custom version of Caddy with the plugins I needed using xcaddy. In case it's helpful, see https://github.com/NixOS/nixpkgs/pull/191883#issuecomment-1250652290.

[^1]: Even opinions about the fixed derivation used by buildGoModule vary, see https://github.com/NixOS/nix/issues/2270.

[winterqt]

@tjni xcaddy requires the network to build, at least with how you have it setup in your derivation. Do you have sandboxing disabled? Otherwise, I don't see how your derivation would work.

On a related note, I've been tossing around an idea for a more reproducible way to build Caddy with plugins. Stay tuned, I guess? (If I can get it to work, that is.)

[tjni]

Yes, I have sandboxing disabled. If I turn it on, I can also abuse fixed derivations. I'm looking forward to what you're trying!

[winterqt]

let
  caddySrc = srcOnly (fetchFromGitHub {
    owner = "caddyserver";
    repo = "caddy";
    rev = "v2.6.0";
    hash = "sha256-BZGfYpQM5e+/LrGqs1oms/vOp9q4UY/ZyDODEsi/wl8=";
  });

  pluginSrc = srcOnly (fetchFromGitHub {
    owner = "porech";
    repo = "caddy-maxmind-geolocation";
    rev = "89d86498ab7d55c9212c0c6b4d1ac9026929147b";
    hash = "sha256-VDb2qwWFjMuGSOIiwuPsQJdj85lx7G59h2w526JY8yM=";
  });

  combinedSrc = stdenv.mkDerivation {
    name = "caddy-src";

    nativeBuildInputs = [ go ];

    buildCommand = ''
      export GOCACHE="$TMPDIR/go-cache"
      export GOPATH="$TMPDIR/go"

      mkdir -p "$out/ourcaddy"

      cp -r ${caddySrc} "$out/caddy"
      cp -r ${pluginSrc} "$out/plugin"

      cd "$out/ourcaddy"

      go mod init caddy
      echo "package main" >> main.go
      echo 'import caddycmd "github.com/caddyserver/caddy/v2/cmd"' >> main.go
      echo 'import _ "github.com/porech/caddy-maxmind-geolocation"' >> main.go
      echo "func main(){ caddycmd.Main() }" >> main.go
      go mod edit -require=github.com/caddyserver/caddy/v2@v2.0.0
      go mod edit -replace github.com/caddyserver/caddy/v2=../caddy
      go mod edit -require=github.com/porech/caddy-maxmind-geolocation@v0.0.0
      go mod edit -replace github.com/porech/caddy-maxmind-geolocation=../plugin
    '';
  };
in
buildGoModule {
  name = "meowdy";

  src = combinedSrc;

  vendorHash = "sha256-6sAZDBH7PlA1S1hKoEVPwKRQPDZ4gb8Ezu/XhT2zIJg=";

  overrideModAttrs = _: {
    postPatch = "cd ourcaddy";

    postConfigure = ''
      go mod tidy
    '';

    postInstall = ''
      mkdir -p "$out/.magic"
      cp go.mod go.sum "$out/.magic"
    '';
  };

  postPatch = "cd ourcaddy";

  postConfigure = ''
    cp vendor/.magic/go.* .
  '';
}

So here's a hacky proof of concept.

My thesis is that as long as the sources are the same, Go should produce the exact same go.mod and go.sum (assuming the Go version doesn't make those change, which has happened in the past). So this should be no less reproducible than any other Go package.

The ergonomics could potentially be improved (using go get ...@commit for example), but for that point specifically: I think fetching the sources through our fetchers is just one more thing we can guarantee to be reliable in terms of reproducibility, if that makes sense.

I welcome any questions/comments/suggestions/counterarguments/etc., as I'm not sure if this is fit for inclusion in Nixpkgs 😅

[tjni]

I think your strategy looks like it can work, introduces no new fixed derivations and no new calls to download dependencies using go. I just don't know enough about Go myself to understand the trade-offs here either 😅

[winterqt]

and no new calls to download dependencies using go

FWIW, I think go mod tidy may download dependencies (could be wrong, can't check right now), but that doesn't affect the output considering everything in Caddy and the plugins are locked with go.sum.

I'm going to try to polish this up when I get a chance and can ask some more knowledgable people to take a look at that point.

[tjni]

@winterqt

I was perusing xcaddy's code, and this part looks interesting to me:

https://github.com/caddyserver/xcaddy/blob/master/environment.go#L142

It seems to follow a similar strategy to yours, so it might be possible to fetch modules locally and use xcaddy to build it without accessing the network.

Though, I see that the implication of the following lines need to be considered:

// doing an empty "go get -d" can potentially resolve some
// ambiguities introduced by one of the plugins;
// see https://github.com/caddyserver/xcaddy/pull/92
err = env.execGoGet(ctx, "", "", "", "")

[winterqt]

It seems to follow a similar strategy to yours, so it might be possible to fetch modules locally and use xcaddy to build it without accessing the network.

Considering all xcaddy does in the end is this (which is exactly where I took this from), I don't see the benefits of using it.

Though, I see that the implication of the following lines need to be considered

I'll look into this (see https://github.com/caddyserver/caddy/issues/4331) -- thanks, Go modules.

[winterqt]

The main issue I see with this approach is making users update the hash of all of the dependencies, which probably isn't the best UX. There's not really a way around it, that I can see, though. :/

[tjni]

Are you talking about the combined vendor hash after substituting the plugins?

[winterqt]

Yes.

[diamondburned]

I think it would be neat if we can have a caddyPackages that are symlinkJoined together into one Go vendor derivation to be used by Caddy. Perhaps something like

{
  services.caddy.plugins = with pkgs.caddyPackages; [
    caddy-webdav
    (go-get "github.com/mholt/caddy-webdav" "${lib.fakeSha256}") # arbitrary Go path
  ];
}

[winterqt]

That would require processing the module index within each vendor folder, resolving conflicts somehow, etc... not really easily possible without a lot of work, I think.

[diamondburned]

What conflicts would there be?

[winterqt]

Module version conflicts, as an example. I'd have to see how go mod vendor handles them. Maybe it's easier than I'm thinking, even if it's one big hack (per se).

[diamondburned]

Module version conflicts

I don't think this is hard. We only need to properly resolve vendor/modules.txt to contain all the joined directories.

Generally, Go will use the latest version of a dependency. We might have to also resolve indirect dependencies (e.g. if A imports C v0.1.0 and B imports C v0.1.1) and choose to keep the latest one.

[winterqt]

Did some digging, turns out we apply a patch that allows us to completely nix (sorry) the modules.txt.

So now we just need to symlink the contents of the vendor directories, somehow resolving indirect dependencies along the way, and 🎉

[KFearsoff]

I think CoreDNS is pretty similar to Caddy in terms of how it handles the plugins. Perhaps the PR I made that allows using external plugins in CoreDNS is relevant to this discussion? https://github.com/NixOS/nixpkgs/pull/205649

[winterqt]

It's sadly nothing we haven't already discussed in the reproducibility/UX regard.

[jpds]

The CoreDNS package had #256710 merged which allows pinning the plugin versions.