search

go-acme / lego

Let's Encrypt/ACME client and library written in Go
https://go-acme.github.io/lego/
MIT License
7.91k stars 1.02k forks source link

pdns: API endpoint not at URL root resulting in incorrect URL queried and thus failing with error code 404 #2128

Closed jotasi closed 4 months ago

jotasi commented 7 months ago

Welcome

What did you expect to see?

When requesting a first or renewing an existing certificate via DNS challenge and PowerDNS API with the API endpoint not being located at the URL root (e.g., https://login.udmedia.de/dns/api instead of https://login.udmedia.de/api), the command should run through without an error.

What did you see instead?

When requesting a first or renewing an existing certificate via DNS challenge and PDNS API with the API endpoint not being located at the URL root (e.g., https://login.udmedia.de/dns/api instead of https://login.udmedia.de/api), the command fails with an error code 404, indicating that an unknown URL was queried.

After debugging, this was pinpointed to the URL generated as endpoint for updating within UpdateRecords in providers/dns/pdns/internal/client.go not being the expected https://login.udmedia.de/dns/api/v1/servers/udmedia/zones/example.com. but instead https://login.udmedia.de/dns/api/v1/dns/api/v1/servers/udmedia/zones/example.com. (i.e., with duplicated dns/api/v1/).

The reason for that is that the URL of the host to which the path is appended by joinPath is already containing the /dns but the URL of the zone that is returned by the provider's PDNS compatible API (determined via GetHostedZone) and AFAIU also by PowerDNS itself (tentative as I don't have a PowerDNS installation to try this out with but see, e.g., discussion here) is an absolute URL path also containing the starting /dns. Therefore, /dns of the Host URL is extended by a second /dns from the Zone URL. Furthermore, in joinPath for API version != 0, any path to add that is not starting with /api (which is the case here as the path starts with /dns/api/...) is also prepended by /api/v1. This then results in the superfluous /api/v1/dns being added between the host URL and the Zone URL path.

This could be fixed by adding a second join function (e.g., joinAbsolutePath) that removes any remaining path from the Host URL before joining and (as this seems unnecessary in that case to me, but please correct me there if this is actually needed for, e.g., earlier version of PowerDNS or the API) not trying to guess whether /api/v1 needs to be prepended to the URL path. The function could look somewhat like this:

func (c *Client) joinAbsolutePath(elem ...string) *url.URL {
    p := path.Join(elem...)

    rawHost := *c.Host
    rawHost.Path = ""

    return rawHost.JoinPath(p)
}

and could replace joinPath here and here.

I can confirm that this fixes the issue at least for my provider. If helpful, I can provide a corresponding PR also including some tests of the new function. However, the caveat here is that I have limited experience with go and PowerDNS in general and don't want to inadvertently break something for someone else.

How do you use lego?

Binary

Reproduction steps

Try requesting a first or renewing an existing certificate via DNS challenge and PDNS API (API version v1) with the API endpoint not being located at the URL root (PDNS_API_URL containing a non-empty path, e.g., https://login.udmedia.de/dns instead of https://login.udmedia.de for an API endpoint at https://login.udmedia.de/dns/api instead of https://login.udmedia.de/api).

The command used (and environment variables set (except for the API key), see Logs section.

(See also discussion #2122)

For my understanding of the origin of the error after debugging this, see section on "What did you see instead?".

Version of lego

lego version 4.15.0 linux/386

Logs

Command executed to generate the log: ```shell export PDNS_API_URL=https://login.udmedia.de/dns export PDNS_SERVER_NAME=udmedia export PDNS_API_KEY= export PDNS_API_VERSION=1 # Required as this provider returns not a list but just a single entry when querying the version ./lego --accept-tos --path '.' -d 'some_domain.de' --email 'some_email@example.xom' --key-type 'ec256' --dns 'pdns' --dns.resolvers '5.1.66.255:53' --server 'https://acme-staging-v02.api.letsencrypt.org/directory' run ``` ```console 2024/02/28 21:46:31 [INFO] [some_domain.de] acme: Obtaining bundled SAN certificate 2024/02/28 21:46:31 [INFO] [some_domain.de AuthURL: https://acme-staging-v02.api.letsencrypt.org/acme/authz-v3/ 2024/02/28 21:46:31 [INFO] [some_domain.de] acme: Could not find solver for: tls-alpn-01 2024/02/28 21:46:31 [INFO] [some_domain.de] acme: Could not find solver for: http-01 2024/02/28 21:46:31 [INFO] [some_domain.de] acme: use dns-01 solver 2024/02/28 21:46:31 [INFO] [some_domain.de] acme: Preparing to solve DNS-01 2024/02/28 21:46:32 [INFO] [some_domain.de] acme: Cleaning DNS-01 challenge 2024/02/28 21:46:32 [WARN] [some_domain.de] acme: cleaning up failed: pdns: unexpected status code: [status code: 404] body: { "error": "Not found" } 2024/02/28 21:46:32 [INFO] Deactivating auth: https://acme-staging-v02.api.letsencrypt.org/acme/authz-v3/ 2024/02/28 21:46:32 Could not obtain certificates: error: one or more domains had a problem: [some_domain.de] [some_domain.de] acme: error presenting token: pdns: unexpected status code: [status code: 404] body: { "error": "Not found" } ```

Go environment (if applicable)

```console $ go version && go env # paste output here ```
ldez commented 7 months ago

Hello,

I'm not sure to understand your problem.

As you can see we have dedicated tests on joinPath: https://github.com/go-acme/lego/blob/82e9a5e2a917e72870781d80d2de515c27208304/providers/dns/pdns/internal/client_test.go#L66

Your implementation will break everything.

And the tests seems to say the opposite of the behavior you described :thinking: . https://github.com/go-acme/lego/blob/82e9a5e2a917e72870781d80d2de515c27208304/providers/dns/pdns/internal/client_test.go#L74-L80

jotasi commented 7 months ago

Hi,

Thanks for the quick reply!

Please find a bit more concrete explanation below.

There are four places where joinPath is used (which just adds the path to the PDNS_API_URL, potentially adding /api/v1 to the front if the provided path does not start with /api and if the API version is not 0):

With a relative path (well represented by the tests and functional in my use case): https://github.com/go-acme/lego/blob/82e9a5e2a917e72870781d80d2de515c27208304/providers/dns/pdns/internal/client.go#L55-L58 With a relative path (well represented by the tests and functional in my use case): https://github.com/go-acme/lego/blob/82e9a5e2a917e72870781d80d2de515c27208304/providers/dns/pdns/internal/client.go#L84-L87 With an absolute path (not represented by the tests and not functional in my use case): https://github.com/go-acme/lego/blob/82e9a5e2a917e72870781d80d2de515c27208304/providers/dns/pdns/internal/client.go#L119-L122 With an absolute path (not represented by the tests and not functional in my use case): https://github.com/go-acme/lego/blob/82e9a5e2a917e72870781d80d2de515c27208304/providers/dns/pdns/internal/client.go#L135-L140 Only the first two are used with a ("manually" built) path that is relative to the APIs root path (`/api` and `/servers/zones/`. These are represented well in, e.g., the test case(s) you have included above and also work for my use case.

The problem is with the latter two usages. Here, the path that is passed to joinPath is starting with zone.URL, where zone is the parsed return of the API call within GetHostedZone (see second code block above). However, at least for my provider, the URL that is returned by querying, in my case, https://login.udmedia.de/dns/api/v1/servers/udmedia/zones/example.com. is an absolute path to the host's root and thus also includes the relative path to the API again that is aready part of PDNS_API_URL (i.e., starting with /dns below rather than /api/v1/...):

{
    "id": "example.com.",
    "url": "\/dns\/api\/v1\/servers\/udmedia\/zones\/example.com.",
    "name": "example.com.",
    "type": "Zone",
<snip>

When this path is then passed to joinPath, this leads to URL with the duplicated /dns/api/v1/dns/api/v1 that I have listed in my original post.

My crude solution suggested above for the 3rd and 4th usage above (NOT replacing joinPath for the first two) works for my case and should also work for the standard case, where the API endpoint sits at the URL root with API version 1 where the /api/v1 should already be part of zone.URL.

However, there are also possibly less invasive solutions. If any of them sounds reasonable, please let me know and I can potentially flesh them out some more: