search

noahkw / acmetk

ACME Toolkit
MIT License
19 stars 2 forks source link

problem adding support for OVH #64

Open m-martin-78 opened 1 year ago

m-martin-78 commented 1 year ago

Hi,

I tried to allow support of OVH API as a DNS provider, so I created the OVH plugin based on the Infoblox class:

@PluginRegistry.register_plugin("ovh")
class OvhClient(ChallengeSolver):
    """OVH DNS-01 challenge solver.

    This challenge solver connects to the OVH API to provision
    DNS TXT records in order to complete the ACME DNS-01 challenge type.
    """

    SUPPORTED_CHALLENGES = frozenset([ChallengeType.DNS_01])
    """The types of challenges that the solver supports."""

    POLLING_DELAY = 1.0
    """Time in seconds between consecutive DNS requests."""

    POLLING_TIMEOUT = 60.0 * 5
    """Time in seconds after which placing the TXT record is considered a failure."""

    DEFAULT_DNS_SERVERS = ["1.1.1.1", "8.8.8.8"]
    """The DNS servers to use if none are specified during initialization."""

    def __init__(self, *, ovh_endpoint, ovh_application_key, ovh_application_secret, ovh_consumer_key, dns_servers=None):
        self._client = ovh.Client(
            endpoint = ovh_endpoint,
            application_key = ovh_application_key,
            application_secret = ovh_application_secret,
            consumer_key = ovh_consumer_key,
        )
        self._psl = PSL()
        self._loop = asyncio.get_event_loop()

        self._resolvers = []

        for nameserver in dns_servers or self.DEFAULT_DNS_SERVERS:
            resolver = dns.asyncresolver.Resolver()
            resolver.nameservers = [nameserver]
            self._resolvers.append(resolver)

    async def connect(self):
        """Connects to the InfoBlox API.

        This method must be called before attempting to complete challenges.
        """
        return

    async def set_txt_record(self, name: str, text: str, views=None, ttl: int = 60):
        """Sets a DNS TXT record.

        :param name: The name of the TXT record.
        :param text: The text of the TXT record.
        :param views: List of views to set the TXT record in. Defaults to *Intern* and *Extern*.
        :param ttl: Time to live of the TXT record in seconds.
        """
        logger.debug("Setting TXT record %s = %s, TTL %d", name, text, ttl)
        apex = self._psl.private_suffix(name)
        privatepart = '.'.join(self._psl.privateparts(name)[:-1])
        # create record
        res = self._client.post(f'/domain/zone/{apex}/record',
            fieldtype="TXT",
            subDomain=privatepart,
            target=text,
            TTL=60)
        # apply modifications to zone (publish record)
        res = self._client.post(f'/domain/zone/{apex}/refresh')

    async def delete_txt_record(self, name: str, text: str):
        """Deletes a DNS TXT record.

        :param name: The name of the TXT record to delete.
        :param text: The text of the TXT record to delete.
        """
        logger.debug("Deleting TXT record %s = %s", name, text)
        apex = self._psl.private_suffix(name)
        privatepart = '.'.join(self._psl.privateparts(name)[:-1])
        # find record
        res = self._client.get(f'/domain/zone/{apex}/record',
            fieldtype="TXT",
            subDomain=privatepart)
        candidates_ids = [i for i in res]
        corresponding_id = None
        for i in candidates_ids:
            res = self._client.get(f'/domain/zone/{apex}/record/{i}')
            if res['target'] == text:
                corresponding_id = i
        if corresponding_id is None:
            raise ValueError('Could not find matching record to delete.')
        # delete record
        res = self._client.delete(f'/domain/zone/{apex}/record/{corresponding_id}')
        # apply modifications to zone (publish record)
        res = self._client.post(f'/domain/zone/{apex}/refresh')

    async def query_txt_record(
        self, resolver: dns.asyncresolver.Resolver, name: str
    ) -> typing.Set[str]:
        """Queries a DNS TXT record.

        :param name: Name of the TXT record to query.
        :return: Set of strings stored in the TXT record.
        """
        txt_records = []

        with contextlib.suppress(
            dns.asyncresolver.NXDOMAIN, dns.asyncresolver.NoAnswer
        ):
            resp = await resolver.resolve(name, "TXT")

            for records in resp.rrset.items.keys():
                txt_records.extend([record.decode() for record in records.strings])

        return set(txt_records)

    async def _query_until_completed(self, name, text):
        while True:
            record_sets = await asyncio.gather(
                *[self.query_txt_record(resolver, name) for resolver in self._resolvers]
            )

            # Determine set of records that has been seen by all name servers
            seen_by_all = set.intersection(*record_sets)

            if text in seen_by_all:
                return

            logger.debug(
                f"{name} does not have TXT {text} yet. Retrying (Records seen by all name servers: {seen_by_all}"
            )
            logger.debug(f"Records seen: {record_sets}")
            await asyncio.sleep(1.0)

    async def complete_challenge(
        self,
        key: josepy.jwk.JWK,
        identifier: acme.messages.Identifier,
        challenge: acme.messages.ChallengeBody,
    ):
        """Completes the given DNS-01 challenge.

        This method provisions the TXT record needed to complete the given challenge.
        Then it polls the DNS for up to :attr:`POLLING_TIMEOUT` seconds to ensure that the record is visible
        to the remote CA's DNS.

        :param key: The client's account key.
        :param identifier: The identifier that is associated with the challenge.
        :param challenge: The challenge to be completed.
        :raises: :class:`~acmetk.client.exceptions.CouldNotCompleteChallenge`
            If the challenge completion attempt failed.
        """
        name = challenge.chall.validation_domain_name(identifier.value)
        text = challenge.chall.validation(key)

        try:
            await self.set_txt_record(name, text)
        except Exception as e:
            logger.exception(
                "Could not set TXT record to solve challenge: %s = %s", name, text
            )
            raise CouldNotCompleteChallenge(
                challenge,
                acme.messages.Error(typ="infoblox", title="error", detail=str(e)),
            )

        # Poll the DNS until the correct record is available
        try:
            await asyncio.wait_for(
                self._query_until_completed(name, text), self.POLLING_TIMEOUT
            )
        except asyncio.TimeoutError:
            raise CouldNotCompleteChallenge(
                challenge,
                acme.messages.Error(
                    typ="infoblox",
                    title="error",
                    detail="Could not complete challenge due to a DNS polling timeout",
                ),
            )

    async def cleanup_challenge(
        self,
        key: josepy.jwk.JWK,
        identifier: acme.messages.Identifier,
        challenge: acme.messages.ChallengeBody,
    ):
        """Performs cleanup for the given challenge.

        This method de-provisions the TXT record that was created to complete the given challenge.

        :param key: The client's account key.
        :param identifier: The identifier that is associated with the challenge.
        :param challenge: The challenge to clean up after.
        """
        name = challenge.chall.validation_domain_name(identifier.value)
        text = challenge.chall.validation(key)

        await self.delete_txt_record(name, text)

My problem is that when my Caddy webserver is pointed to the acme server (I removed the reverse-proxy as it cannot use acmetk itself to get a certificate), I keep getting a problem about deserializing "status" in JWTs:

2023-08-18 16:42:47,954 - aiohttp.access - INFO - 10.0.2.100 [18/Aug/2023:14:42:47 +0000] "POST /new-order HTTP/1.1" 400 389 "-" "Caddy/ CertMagic acmez (linux; amd64)"
2023-08-18 16:42:48,222 - aiohttp.server - ERROR - Error handling request
Traceback (most recent call last):
  File "/usr/local/lib/python3.9/site-packages/josepy/json_util.py", line 321, in fields_from_json
    fields[slot] = field.decode(value)
  File "/usr/local/lib/python3.9/site-packages/josepy/json_util.py", line 110, in decode
    return self.fdec(value)
  File "/usr/local/lib/python3.9/site-packages/acme/messages.py", line 87, in from_json
    raise jose.DeserializationError(f'{cls.__name__} not recognized')
josepy.errors.DeserializationError: Deserialization error: Status not recognized

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.9/site-packages/aiohttp/web_protocol.py", line 433, in _handle_request
    resp = await request_handler(request)
  File "/usr/local/lib/python3.9/site-packages/aiohttp/web_app.py", line 504, in _handle
    resp = await handler(request)
  File "/usr/local/lib/python3.9/site-packages/aiohttp/web_middlewares.py", line 117, in impl
    return await handler(request)
  File "/app/acmetk/server/server.py", line 1253, in error_middleware
    response = await handler(request)
  File "/app/acmetk/server/server.py", line 1216, in host_ip_middleware
    return await handler(request)
  File "/app/acmetk/server/server.py", line 1243, in aiohttp_jinja2_middleware
    return await handler(request)
  File "/app/acmetk/server/server.py", line 1567, in new_order
    obj = acme.messages.NewOrder.json_loads(jws.payload)
  File "/usr/local/lib/python3.9/site-packages/josepy/interfaces.py", line 177, in json_loads
    return cls.from_json(loads)
  File "/usr/local/lib/python3.9/site-packages/josepy/json_util.py", line 331, in from_json
    return cls(**cls.fields_from_json(jobj))
  File "/usr/local/lib/python3.9/site-packages/josepy/json_util.py", line 323, in fields_from_json
    raise errors.DeserializationError('Could not decode {0!r} ({1!r}): {2}'.format(
josepy.errors.DeserializationError: Deserialization error: Could not decode 'status' (''): Deserialization error: Status not recognized

I also tried updating josepy and all the other dependences of the project, but I still have the same problem.

commonism commented 1 year ago

The error is unrelated to your ovh dns01 validation.

https://github.com/noahkw/acmetk/blob/4bf6202babbfa1cf91801a8f1bd3ae3a02737799/acmetk/server/server.py#L744-L754 The message sent by caddy is not valid. Deserialization of the status field fails, "" is not recognized as valid value. https://datatracker.ietf.org/doc/html/rfc8555#section-7.1.3 https://github.com/certbot/certbot/blob/b1978ff18837e40d16eedf2090330af53d8ceaa5/acme/acme/messages.py#L207-L219 https://github.com/certbot/certbot/blob/b1978ff18837e40d16eedf2090330af53d8ceaa5/acme/acme/messages.py#L613-L681

I'd look into using lexicon to get ovh dns01 validation, maybe pick up my PR? https://github.com/noahkw/acmetk/pull/63/files

That said - currently I'd consider this code base unmaintained.

m-martin-78 commented 1 year ago

@commonism: Thanks for your help! considering the error message, I indeed think that the problem comes from the Caddy ACME client (or the parser if a null value is valid according to the protocol) and not from the OVH challenge solver as it is not even called at this point. I'll try your PR this week I hope.

I understand this project is unmaintained, the latest commit is two years old, yet the need is still there: allowing internal resources to obtain LE certificates without giving each of them the keys to the public DNS zone. Do you know of an alternative project that does that?

commonism commented 1 year ago

Your uscase is exactly what this project was meant to cover. I'd go with it.

For your debug - have caddy access the acme service via http, capture stream or dump the data in the service Share the capture, I'll have a look. But I think I 'll have to refer to caddy, and I I'd expect them to refer you to acmez.

commonism commented 1 year ago

In case you are still interested … it's in progress.