route53 support

[dkebler]

I see at this time you don't want to support route53 DNS challenge #45

I have my own custom route53 caddy docker container, and it runs great and only requires I provide the two credentials for aws.

AWS_ACCESS_KEY_ID AWS_SECRET_ACCESS_KEY

all other "settings" needed for access are done within AWS with an IAM security profile.

You say it takes too many "settings" which I don't understand. Are you not using the route53 plugin?

https://github.com/caddy-dns/route53

with latest versions of Caddy you can add such a plugin "on the fly" instead of having to build a custom bin file so maybe that would make it easier to support other dns providers (other caddy plugins)

https://caddyserver.com/docs/command-line#caddy-add-package

Since I am very familiar with route53 maybe I can help. Your plugin is great, but without support for route53 I won't be able to use it short of forking your repo and trying to custom code it.

[dkebler]

The caddyfile for route53 reverse proxy is pretty simple

(r53) {
  tls someone.net@gmail.com {
    dns route53 {
      max_retries 10
    }
  }
}

https://docker.mydomain.net
{
   import r53
   reverse_proxy /* http://server.mydomain.net:9000
 }

[Monviech]

Currently, adding a new DNS Provider is the most tedious task.

Using caddy add-package doesn't work, because the combination of modules won't allow the build to complete. xcaddy is mandatory and build errors have to be fixed in the source code. https://github.com/Monviech/os-caddy-plugin/blob/main/usr/local/bin/README.md

The template is already pretty big and a DNS Provider needs to be added in the Dynamic DNS and the TLS Macro to be generated at the right spot in the Caddyfile - if it needs a custom way to be generated. https://github.com/Monviech/os-caddy-plugin/blob/main/usr/plugins/devel/caddy/src/opnsense/service/templates/Pischem/Caddy/Caddyfile

Right now, most domain providers are configured like this, for that you only need to add the dns provider name to the caddy.xml and it already works.

https://github.com/Monviech/os-caddy-plugin/blob/32c52bd449ca392cc43cc3418cda12a2829f6d52/usr/plugins/devel/caddy/src/opnsense/mvc/app/models/Pischem/Caddy/Caddy.xml#L24

Heres an example how almost all dns providers are configured in a generated Caddyfile:

# Global Options
{
        dynamic_dns {
                provider duckdns API_KEY
                domains {
                        domain.duckdns.com @
                }
        }
}
# Reverse Proxy
domain.duckdns.com {
        tls {
                dns duckdns API_KEY
        }
        handle {
                reverse_proxy 172.16.0.173 {
                }
        }
}

There are two exceptions to this rule, desec and porkbun. Porkbun has an own second API field. So it has a custom section in the template generation.

If route53 is another exception to that rule, it would take some effort to include. Can you provide an example for route53 using the exact template I have provided as example?

[dkebler]

I am not familiar with nor have ever had need of the dynamic_dns directive.

Is that something you created (a module) or this it this package/module https://caddyserver.com/docs/modules/dynamic_dns that you added with xcaddy?

I use the route53 plugin only so I can get certs for subdomins only valid (can be looked up) within my LAN so maybe this project just is not going to work for me assuming it requires A/C records upstream at the primary DNS provider (e.g. route53)

[dkebler]

Maybe the route to go is to make a one off versions instead of trying to incorporate all DNS providers. Maybe that would cause less problems with xcaddy builds.

Honestly I know nothing about how to make a package for opnsense so I have no feel for the efforts involved or what kind of scripting/templates are involved.

I am probably just not going to go there when I can right now just direct opnsense firewall ports 80 and 443 to a machine inside my LAN that is running caddy (my container) which is what I do now. I was kinda excited that maybe I could do that on the opnsense box but I understand that it is too much work to support all DNS providers. So I'll just close this.

But if it does end up being supported for my use case a heads up would be appreciated.

[Monviech]

In the plugin, a supported DNS Provider does have to support these features, which means:

• Dynamic DNS (with the mholt dynamic-dns module)
• ACME TXT Challenge (Setting TXT Records so Lets Enxrypt can issue a cert)

Depening on what a user selects, the right options have to be generated in the Caddyfile. Thats why I wanted to know how it should look properly for route53 if all features are selected.

Since you cant tell me and I dont have time to set up accounts and test it all myself, I can't add the logic for it. If you still have time, please tell me how the provided Caddyfile should look like for route53.

Also making many one off versions would mean if theres an update I have to maintain and rebuild like 10-15 differend combinations.of the binary and the plugin packages.

[Monviech]

Here's another example, how the Caddyfile looks with porkbun when dynamic DNS and DNS-01 TXT Challenge is activated:

# Global Options
{
        # User checked "Dynamic DNS" checkbox in "Edit Reverse Proxy Domain"
        dynamic_dns {
                provider porkbun {
                        api_key 23987gh234q78rgw2q3o84
                        api_secret_key 78923gBAUHZSFGo83
                }
                domains {
                        example.com @
                }
        }
}

# Reverse Proxy Configuration
example.com {
        tls {
                # User checked "DNS-01 challenge" checkbox in "Edit Reverse Proxy Domain"
                dns porkbun {
                        api_key 23987gh234q78rgw2q3o84
                        api_secret_key 78923gBAUHZSFGo83
                }
        }
        handle {
                reverse_proxy 192.168.1.1 {
                }
        }
}

[dkebler]

Like I said I have not used that module/package to do DNS "text" challenge. So I will have to spin something up and try it.

I always assumed that there was no "generic" DNS challenge modules (at least that was true awhile back) so I never went looking for it, but maybe this mholt's dynamic_dns module is such?? Maybe I'll ask at caddy forum.

So go ahead and leave this open and eventually I'll get back to it once I understand more although I can say that the porkbun caddyfile has the "same" two credentials as route53 so maybe it's just

        tls {
                # User checked "DNS-01 challenge" checkbox in "Edit Reverse Proxy Domain"
                dns route53 {
                        api_key <some aws IAM key>
                        api_secret_key <some aws IAM secret >
                }

So what is not clear is if that will then call the correct functions at AWS in order to get the record written, checked and removed. That only seems possible if the dynamic_dns module includes some route53 specific code (i.e. dns route53 { ) like the route53 dns module has.

Can we get mholt to comment?

[Monviech]

The dynamic_dns module just needs the same credentials as the tls directive for the Let's Encrypt certificates.

So if the credentials work here (in the example below) that's all I need to know. If you can test out what exactly has to be there, then I can think about adding it.

# Reverse Proxy Configuration
example.com {
        tls {
                # User checked "DNS-01 challenge" checkbox in "Edit Reverse Proxy Domain"
                dns route53 {
                        access_key_id 23987gh234q78rgw2q3o84
                        secret_access_key 78923gBAUHZSFGo83
                }
        }
        handle {
                reverse_proxy 192.168.1.1 {
                }
        }
}

The official module states it needs this, but it also says "OPTIONAL", so you have to tell me what really is needed.

tls {
  dns route53 {
    max_retries 10 // optional
    aws_profile "real-profile" // optional
    access_key_id "AKI..." // optional
    secret_access_key "wJa..." // optional
    token "TOKEN..." // optional
    region "us-east-1" // optional
  }
}

And judging from your snippet you provided earlier, you are using the ACME DNS-01 Challenge to get the certificate from Let's Encrypt for your subdomains. Otherwise, I don't understand what it does.

[dkebler]

In my original example that assumes one has the access key (secret and id) as environment variables

AWS_ACCESS_KEY_ID
AWS_SECRET_ACCESS_KEY

but alternatively you can hard code them exactly as you wrote or as environment variables instead of hard coded values

                dns route53 {
                        access_key_id 23987gh234q78rgw2q3o84
                        secret_access_key 78923gBAUHZSFGo83
                }
or
          dns route53 {
                        access_key_id $MYID
                        secret_access_key $MYSECRET
                }

In general that is all AWS needs. The access key id is associated with an AWS IAM user who have been given the particular read/write permissions on the DNS domain records. The remaining ones are optional. So what you are maybe saying is that I should just try it and it might work as is? I have two live installs I really don't want to use for testing so I'll have to spin up something might be several days b4 I get to this.

[Monviech]

Thanks for the info. It would be great if you can test it. You dont need a seperate instance when you have an opnsense running. Just use my plugin and use the import feature: https://github.com/Monviech/os-caddy-plugin?tab=readme-ov-file#custom-configuration-files

Create a random subdomain and put the configuration into a "test.conf" file just like here https://github.com/Monviech/os-caddy-plugin/issues/84#issuecomment-1920729954

Then go into the GUI and look into the logfile if it issued the certificate. Thank you :)

[Monviech]

I'll create a test package in a few days with route 53 included.

[Monviech]

I've built new packages as promised. You can try them out by downloading them onto your firewall and then using pkg install ./caddy-2.7.6_3.pkg and pkg install ./os-caddy-1.4.3.pkg inside the folder you downloaded them to.

caddy-2.7.6_3.pkg os-caddy-1.4.3.pkg

[Monviech]

It's now included in the normal repository, so it comes automatically as update.

[Monviech]

https://github.com/Monviech/os-caddy-plugin/commit/dd2a6741c375b44e65bd3c6558041c7a1440eaa2

[Monviech]

@dkebler Did you have time to try it out yet?