Unable to update challenge :: authorization must be pending

[sierramike]

Hello,

I'm searching what I'm doing wrong. I wrote the following simple code based on the base documentation of this library, and started testing (using the WellKnownServers.LetsEncryptStagingV2 which is returned by a property I called "Server").

        AcmeContext acme;
        IAccountContext account;

        if (File.Exists("AccountKey.pem"))
        {
            Console.WriteLine("Pem key file found, reading key from file.");

            var pemKey = File.ReadAllText("AccountKey.pem");
            var accountKey = KeyFactory.FromPem(pemKey);
            acme = new AcmeContext(Server, accountKey);
            account = await acme.Account();

            Console.WriteLine("Login done.");
        }
        else
        {
            Console.WriteLine("No pem key file found, creating account.");

            acme = new AcmeContext(Server);
            account = await acme.NewAccount("mymail@gmail.com", true);

            Console.WriteLine("Account created.");

            var pemKey = acme.AccountKey.ToPem();

            Console.WriteLine("Pem key:");
            Console.WriteLine(pemKey);

            File.WriteAllText("AccountKey.pem", pemKey);

            Console.WriteLine("Pem key file written on disk.");
        }

        var order = await acme.NewOrder(new[] { "*.mydomain.fr" });
        Console.WriteLine($"Order created, location : {order.Location}");

        var authz = (await order.Authorizations()).First();
        Console.WriteLine($"Got authorizations, location : {authz.Location}");

        var dnsChallenge = await authz.Dns();
        var dnsTxt = acme.AccountKey.DnsTxt(dnsChallenge.Token);
        Console.WriteLine($"DNS Challenge text : {dnsTxt}");

        Console.WriteLine("Please update DNS then press a key...");
        Console.ReadKey();

        var v = await dnsChallenge.Validate();

        while(v.Status != Certes.Acme.Resource.ChallengeStatus.Valid)
        {
            Console.WriteLine($"Validation returned status : {v.Status}, retrying in 10 seconds.");
            Thread.Sleep(10000);
            v = await dnsChallenge.Validate();
        }

        Console.WriteLine($"Validation returned status : {v.Status}");

        var privateKey = KeyFactory.NewKey(KeyAlgorithm.ES256);
        var cert = await order.Generate(new CsrInfo { CountryName = "FR", State = "State", Locality = "City", Organization = "Org", CommonName = "*.mydomain.fr" }, privateKey);
        var certPem = cert.ToPem();

        File.WriteAllText("cert.pem", certPem);

        Console.WriteLine($"Certificate PEM written to file:\n{certPem}");

        var pfxBuilder = cert.ToPfx(privateKey);
        var pfx = pfxBuilder.Build("my-cert", "abcd1234");

        File.WriteAllBytes("cert.pfx", pfx);

        Console.WriteLine($"PFX certificate written to file.");

        Console.WriteLine($"Press a key...");
        Console.ReadKey();

This code runs correctly until the DNS challenge key which is returned correctly. I manually add the challenge key to the DNS, then press my key so it tries to Validate().

The validation returns "Pending", then waits 10 seconds, then throws an exception (on the next Validate()): AcmeRequestException : Fail to load resource from 'https://acme-staging-v02.api.letsencrypt.org/acme/chall-v3/2031491498/M1rHWg'. urn:ietf:params:acme:error:malformed: Unable to update challenge :: authorization must be pending

The fact is, I first tried to develop a more complex code and placed multiple values in the order : mydomain.fr, myotherdomain.com, .mydomain.fr, .myotherdomain.com to have SAN in the cert, and got the exact same issue : first pass on validate returns an object with "Pending" status, but next pass will throw this exception.

Suspected an issue with my DNS, but I tried with "www.shieldsigned.com" website which I was using until now for my certs, and managed to generate successfully the certificate after putting the 4 _acme-challenge values in the domain DNSs. So it seems not related with the DNS configuration.

Anyone can help and tell me what's wrong with my code? Thanks a lot.

(btw, I tried to switch to LetsEncryptV2 instead of staging, but then it failed immediately when trying to place the order, complaining about not authorized action ... but that's another issue it seems ... would like to have it work on the staging environment first ...)

[webprofusion-chrisc]

Hi, you don't need to keep calling Validate on the challenge (just do it once), instead you need to poll the status of the challenge (or the whole order):

 var result = await dnsChallenge.Resource();

  if (result.Status != AuthorizationStatus.Valid && result.Status != AuthorizationStatus.Invalid)
  {
      await Task.Delay(1000);
  }

// then loop while the status is neither Valid or Invalid (e.g. until the challenge either passed or failed)

Regarding switching from staging to the production API, they are different systems so you need a new AccountKey for the real API, e.g. a new account registration and saved key file.

Note also that CsrInfo only needs CommonName set, it's technically invalid to supply the other values because Let's Encrypt can't validate them.

[sierramike]

Thanks a lot for your help, I gave a try to your code, polling dnsChallenge.Resource() every 4 seconds, but it stays "Pending" indefinitely, with the ACME challenge txt record set correctly in the DNS ...

The code I used is :

        var chR = await dnsChallenge.Resource();
        while(chR.Status != Certes.Acme.Resource.ChallengeStatus.Valid && chR.Status != Certes.Acme.Resource.ChallengeStatus.Invalid)
        {
            Console.WriteLine($"dnsChallenge status : {chR.Status}, retry in 4 seconds.");
            Thread.Sleep(4000);
            chR = await dnsChallenge.Resource();
        }
        Console.WriteLine($"dnsChallenge status : {chR.Status}");

[webprofusion-chrisc]

Are you still calling dnsChallenge.Validate() beforehand? That's still required to start off the validation attempt.

[sierramike]

Well, in fact if I fire dnsChallenge.Validate() too early, i.e. before DNS records have time to replicate, the validation process will fail to Invalid status because it doesn't find the TXT challenge record. I was looking for some way to ask the validation process to try validation but not fail. A way to loop until it finds the TXT challenge record instead of failing and needing to place a new order, thus a new challenge text, with again the need to wait until DNS records have replicated across servers ...

[webprofusion-chrisc]

Yes, so you need ensure DNS records are replicated to your nameservers before you attempt .Validate() but after that you're just polling to see if the validation check has been performed (by Let's Encrypt) or not.

[StefH]

@sierramike I had the same issue, so I just used this NuGet library (https://dnsclient.michaco.net/) to query the TXT record.

Example code:

var result = await lookup.QueryAsync(dnsChallenge.Record, QueryType.TXT);
var txtRecord = result.Answers.TxtRecords().FirstOrDefault();
var text = txtRecord?.Text.FirstOrDefault();

And once the correct TXT value is returned, I continue with Validate and the next steps.

[sierramike]

Hi, I abandonned this script but now that shieldsigned has closed I am in the need to automate by myself, thus coming back on this. I started testing again, and here is where I'm stuck : 1) I order a certificate with a list of domains, with and without wildcards : domain1.com, domain2.com, .domain1.com, .domain2.com 2) I get the ACME challenges for each (4 challenges) 3) I put the challenges in the DNS TXT records (2 records for each domain for both challenges, that was working fine when using shieldsigned) 4) Challenge check becomes valid for wildcard domains but stays pending for root domain (aka domain1.com and domain2.com)

I went to the order URI (https://acme-v02.api.letsencrypt.org/acme/order/xxxxxxxxxxxxx/xxxxxxxxxxxxx" which returns a JSON data containing a list of authorizations URLs, one for each domain, addresses are like "https://acme-v02.api.letsencrypt.org/acme/authz-v3/xxxxxxxxxxxxxxx" I visited all these URIs and I get : For wildcard domains :

{
  "identifier": {
    "type": "dns",
    "value": "domain1.com"
  },
  "status": "valid",
  "expires": "2024-01-03T00:42:46Z",
  "challenges": [
    {
      "type": "dns-01",
      "status": "valid",
      "url": "https://acme-v02.api.letsencrypt.org/acme/chall-v3/xxxxxxxxxxxxxxx/aq_x8Q",
      "token": "04DBRJnHKHgIxxxxxxxxxxxxxxxxxHea_XLvI-XnqEo",
      "validationRecord": [
        {
          "hostname": "domain1.com"
        }
      ],
      "validated": "2023-12-04T00:42:45Z"
    }
  ],
  "wildcard": true
}

But for non wildcard domains I get :

{
  "identifier": {
    "type": "dns",
    "value": "domain1.com"
  },
  "status": "pending",
  "expires": "2023-12-11T00:50:21Z",
  "challenges": [
    {
      "type": "http-01",
      "status": "pending",
      "url": "https://acme-v02.api.letsencrypt.org/acme/chall-v3/xxxxxxxxxxxxxx/Or8HNg",
      "token": "LWu2ovPE-OXdLxxxxxxxxxxxxxxxxxxoivJ_w6eSzde8_Vw"
    },
    {
      "type": "dns-01",
      "status": "pending",
      "url": "https://acme-v02.api.letsencrypt.org/acme/chall-v3/xxxxxxxxxxxxxx/1V9Ejw",
      "token": "LWu2ovPE-OXdLxxxxxxxxxxxxxxxxxxoivJ_w6eSzde8_Vw"
    },
    {
      "type": "tls-alpn-01",
      "status": "pending",
      "url": "https://acme-v02.api.letsencrypt.org/acme/chall-v3/xxxxxxxxxxxxxx/2kYHUA",
      "token": "LWu2ovPE-OXdLxxxxxxxxxxxxxxxxxxoivJ_w6eSzde8_Vw"
    }
  ]
}

Seems like if LE was waiting for http-01 validation and is not checking the dns-01 validation.

If I force validation by calling the Validate() method, it fails for non wildcard challenges, the JSON data shows :

{
  "identifier": {
    "type": "dns",
    "value": "domain1.com"
  },
  "status": "invalid",
  "expires": "2023-12-11T00:50:21Z",
  "challenges": [
    {
      "type": "http-01",
      "status": "invalid",
      "error": {
        "type": "urn:ietf:params:acme:error:unauthorized",
        "detail": "12.34.56.78: Invalid response from http://domain1.com/.well-known/acme-challenge/LWu2ovPE-OXdLxxxxxxxxxxxxxxxxxxoivJ_w6eSzde8_Vw: 404",
        "status": 403
      },
      "url": "https://acme-v02.api.letsencrypt.org/acme/chall-v3/xxxxxxxxxxxxxx/Or8HNg",
      "token": "LWu2ovPE-OXdLxxxxxxxxxxxxxxxxxxoivJ_w6eSzde8_Vw",
      "validationRecord": [
        {
          "url": "http://domain.1com/.well-known/acme-challenge/LWu2ovPE-OXdLxxxxxxxxxxxxxxxxxxoivJ_w6eSzde8_Vw",
          "hostname": "domain1.com",
          "port": "80",
          "addressesResolved": [
            "12.34.56.78"
          ],
          "addressUsed": "12.34.56.78"
        }
      ],
      "validated": "2023-12-04T10:52:17Z"
    }
  ]
}

Which seems as if it only checks for http-01 validation. How is it possible to force dns-01 validation ?