Add DNS provider for HuaweiCloud

endymx commented 3 weeks ago

```console HUAWEICLOUD_ACCESS_KEY_ID=your-access-key-id \ HUAWEICLOUD_SECRET_ACCESS_KEY=your-secret-access-key \ HUAWEICLOUD_REGION=cn-south-1 \ lego --email your@example.com --dns huaweicloud --domains *.example.org run 2024/09/08 16:55:02 No key found for account 345793738@qq.com. Generating a P256 key. 2024/09/08 16:55:02 Saved key to lego2/accounts/acme-v02.api.letsencrypt.org/345793738@qq.com/keys/345793738@qq.com.key 2024/09/08 16:55:04 [INFO] acme: Registering account for 345793738@qq.com !!!! HEADS UP !!!! Your account credentials have been saved in your Let's Encrypt configuration directory at "lego2/accounts". You should make a secure backup of this folder now. This configuration directory will also contain certificates and private keys obtained from Let's Encrypt so making regular backups of this folder is ideal. 2024/09/08 16:55:04 [INFO] [*.dns-test.us.kg] acme: Obtaining bundled SAN certificate 2024/09/08 16:55:05 [INFO] [*.dns-test.us.kg] AuthURL: https://acme-v02.api.letsencrypt.org/acme/authz-v3/4008965549962024/09/08 16:55:05 [INFO] [*.dns-test.us.kg] acme: use dns-01 solver 2024/09/08 16:55:05 [INFO] [*.dns-test.us.kg] acme: Preparing to solve DNS-01 2024/09/08 16:55:07 [INFO] [*.dns-test.us.kg] acme: Trying to solve DNS-01 2024/09/08 16:55:07 [INFO] [*.dns-test.us.kg] acme: Checking DNS record propagation. [nameservers=183.60.83.19:53,183.60.82.98:53] 2024/09/08 16:55:09 [INFO] Wait for propagation [timeout: 1m0s, interval: 2s] 2024/09/08 16:55:19 [INFO] [*.dns-test.us.kg] acme: Waiting for DNS record propagation. 2024/09/08 16:55:32 [INFO] [*.dns-test.us.kg] acme: Waiting for DNS record propagation. 2024/09/08 16:55:44 [INFO] [*.dns-test.us.kg] acme: Waiting for DNS record propagation. 2024/09/08 16:55:56 [INFO] [*.dns-test.us.kg] acme: Waiting for DNS record propagation. 2024/09/08 16:56:13 [INFO] [*.dns-test.us.kg] The server validated our request 2024/09/08 16:56:13 [INFO] [*.dns-test.us.kg] acme: Cleaning DNS-01 challenge 2024/09/08 16:56:14 [INFO] [*.dns-test.us.kg] acme: Validations succeeded; requesting certificates 2024/09/08 16:56:17 [INFO] [*.dns-test.us.kg] Server responded with a certificate. ```

Fixes #1543

ldez commented 3 weeks ago

Hello, in order for a PR adding a DNS provider to be accepted, you have to:

[x] add a description to your PR
[ ] be able to maintain this provider
[x] have a homogeneous design with the other providers
[x] add tests (units)
```
make test
```
[x] add tests ("live") https://github.com/go-acme/lego/blob/c938de68f262c057542d44bcb67ef961cefcc60e/providers/dns/cloudflare/cloudflare_test.go#L125-L151

make test

[x] add a provider descriptor
[x] generate CLI help, documentation, and readme.
```
make generate-dns
```
[x] be able to do: (and put the output of this command to a comment in your PR)
```
rm -rf .lego
```

./lego -m your@email.com --dns YOUR_PROVIDER_NAME -d *.example.com -d example.com -s https://acme-staging-v02.api.letsencrypt.org/directory run

Note the wildcard domain is important.
- [x] pass the linter ([golangci-lint](https://github.com/golangci/golangci-lint#install) must be installed):
```shell
make checks

[x] do go mod tidy

ldez commented 3 weeks ago

Hello,

I'm waiting for the output logs:

rm -rf .lego

./lego -m your@email.com --dns YOUR_PROVIDER_NAME -d *.example.com -d example.com -s https://acme-staging-v02.api.letsencrypt.org/directory run

Notes:

The wildcard domain is important.
This command uses -d *.example.com -d example.com.

endymx commented 3 weeks ago

Hello, the code runs on the current PR

root@VM-20-10-debian:~# rm -rf .lego

HUAWEICLOUD_ACCESS_KEY_ID=GN****EQ HUAWEICLOUD_SECRET_ACCESS_KEY=xfo****zf5 HUAWEICLOUD_REGION=cn-south-1 ./lego -m endymxcn@gmail.com --dns huaweicloud -d *.dns-test.us.kg -d dns-test.us.kg -s https://acme-staging-v02.api.letsencrypt.org/directory run
2024/09/09 15:10:57 No key found for account endymxcn@gmail.com. Generating a P256 key.
2024/09/09 15:10:57 Saved key to /root/.lego/accounts/acme-staging-v02.api.letsencrypt.org/endymxcn@gmail.com/keys/endymxcn@gmail.com.key
2024/09/09 15:10:59 Please review the TOS at https://letsencrypt.org/documents/LE-SA-v1.4-April-3-2024.pdf
Do you accept the TOS? Y/n
Y
2024/09/09 15:11:14 [INFO] acme: Registering account for endymxcn@gmail.com
!!!! HEADS UP !!!!

Your account credentials have been saved in your Let's Encrypt
configuration directory at "/root/.lego/accounts".

You should make a secure backup of this folder now. This
configuration directory will also contain certificates and
private keys obtained from Let's Encrypt so making regular
backups of this folder is ideal.
2024/09/09 15:11:15 [INFO] [*.dns-test.us.kg, dns-test.us.kg] acme: Obtaining bundled SAN certificate
2024/09/09 15:11:17 [INFO] [*.dns-test.us.kg] AuthURL: https://acme-staging-v02.api.letsencrypt.org/acme/authz-v3/13931441993
2024/09/09 15:11:17 [INFO] [dns-test.us.kg] AuthURL: https://acme-staging-v02.api.letsencrypt.org/acme/authz-v3/13931442003
2024/09/09 15:11:17 [INFO] [*.dns-test.us.kg] acme: use dns-01 solver
2024/09/09 15:11:17 [INFO] [dns-test.us.kg] acme: Could not find solver for: tls-alpn-01
2024/09/09 15:11:17 [INFO] [dns-test.us.kg] acme: Could not find solver for: http-01
2024/09/09 15:11:17 [INFO] [dns-test.us.kg] acme: use dns-01 solver
2024/09/09 15:11:17 [INFO] [*.dns-test.us.kg] acme: Preparing to solve DNS-01
panic: assignment to entry in nil map

goroutine 1 [running]:
github.com/go-acme/lego/v4/providers/dns/huaweicloud.(*DNSProvider).Present(0xc0001459c0, {0xc0011fafd0, 0xe}, {0xc000076660, 0x2b}, {0xc0013676e0?, 0x3?})
        github.com/go-acme/lego/v4/providers/dns/huaweicloud/huaweicloud.go:143 +0x2fc
github.com/go-acme/lego/v4/challenge/dns01.(*Challenge).PreSolve(0xc0011a2b40, {{0xc0011fafe0, 0x7}, {0x0, 0xede79d113, 0x0}, {{0xc0011fafc8, 0x3}, {0xc0011fafd0, 0xe}}, ...})
        github.com/go-acme/lego/v4/challenge/dns01/dns_challenge.go:95 +0x27a
github.com/go-acme/lego/v4/challenge/resolver.parallelSolve({0xc0015513a0, 0x2, 0x2}, 0xc001600b40)
        github.com/go-acme/lego/v4/challenge/resolver/prober.go:135 +0x11e
github.com/go-acme/lego/v4/challenge/resolver.(*Prober).Solve(0xc00012c548, {0xc001444dd0, 0x2, 0x14?})
        github.com/go-acme/lego/v4/challenge/resolver/prober.go:84 +0x510
github.com/go-acme/lego/v4/certificate.(*Certifier).Obtain(0xc00158c140, {{0xc000e521c0, 0x2, 0x2}, {0x0, 0x0}, 0x0, {0x0, 0x0, 0x0}, ...})
        github.com/go-acme/lego/v4/certificate/certificates.go:172 +0x402
github.com/go-acme/lego/v4/cmd.obtainCertificate(0xc0011a2280, 0xc001598060)
        github.com/go-acme/lego/v4/cmd/cmd_run.go:199 +0x23c
github.com/go-acme/lego/v4/cmd.run(0xc0011a2280)
        github.com/go-acme/lego/v4/cmd/cmd_run.go:105 +0x257
github.com/urfave/cli/v2.(*Command).Run(0xc000b53a20, 0xc0011a2280, {0xc0010127d0, 0x1, 0x1})
        github.com/urfave/cli/v2@v2.27.2/command.go:276 +0x7e2
github.com/urfave/cli/v2.(*Command).Run(0xc001198000, 0xc0011943c0, {0xc00013e000, 0xc, 0xc})
        github.com/urfave/cli/v2@v2.27.2/command.go:269 +0xa65
github.com/urfave/cli/v2.(*App).RunContext(0xc000f00e00, {0x3bb4230, 0x60f8140}, {0xc00013e000, 0xc, 0xc})
        github.com/urfave/cli/v2@v2.27.2/app.go:333 +0x58b
github.com/urfave/cli/v2.(*App).Run(...)
        github.com/urfave/cli/v2@v2.27.2/app.go:307
main.main()
        ./main.go:42 +0x1c7

I have pushd new code to solve this problem

endymx commented 3 weeks ago

But there are new error, the second round domain DNS Present resolving terminates at line 142:

d.recordIDsMu.Lock()
d.recordIDs[token] = recordSetID
d.recordIDsMu.Unlock()

The second round CleanUp run reports errors:

2024/09/09 15:45:12 [INFO] [*.dns-test.us.kg, dns-test.us.kg] acme: Obtaining bundled SAN certificate
2024/09/09 15:45:14 [INFO] [*.dns-test.us.kg] AuthURL: https://acme-staging-v02.api.letsencrypt.org/acme/authz-v3/13931740453
2024/09/09 15:45:14 [INFO] [dns-test.us.kg] AuthURL: https://acme-staging-v02.api.letsencrypt.org/acme/authz-v3/13931740463
2024/09/09 15:45:14 [INFO] [*.dns-test.us.kg] acme: use dns-01 solver
2024/09/09 15:45:14 [INFO] [dns-test.us.kg] acme: Could not find solver for: tls-alpn-01
2024/09/09 15:45:14 [INFO] [dns-test.us.kg] acme: Could not find solver for: http-01
2024/09/09 15:45:14 [INFO] [dns-test.us.kg] acme: use dns-01 solver
2024/09/09 15:45:14 [INFO] [*.dns-test.us.kg] acme: Preparing to solve DNS-01
2024/09/09 15:45:16 [WARN] presend token: 7Ki11nC8pPJJzxsGF0lAEXLs1AFSd7nKCLfK7RQ91bg
2024/09/09 15:45:16 [INFO] Wait for record set sync on dns-test.us.kg [timeout: 1m0s, interval: 2s]
2024/09/09 15:45:16 [INFO] [dns-test.us.kg] acme: Preparing to solve DNS-01
2024/09/09 15:45:17 [INFO] [*.dns-test.us.kg] acme: Trying to solve DNS-01
2024/09/09 15:45:17 [INFO] [*.dns-test.us.kg] acme: Checking DNS record propagation. [nameservers=183.60.83.19:53,183.60.82.98:53]
2024/09/09 15:45:19 [INFO] Wait for propagation [timeout: 1m0s, interval: 2s]
2024/09/09 15:45:29 [INFO] [*.dns-test.us.kg] acme: Waiting for DNS record propagation.
2024/09/09 15:45:42 [INFO] [*.dns-test.us.kg] acme: Waiting for DNS record propagation.
2024/09/09 15:45:52 [INFO] [*.dns-test.us.kg] The server validated our request
2024/09/09 15:45:52 [INFO] [*.dns-test.us.kg] acme: Cleaning DNS-01 challenge
2024/09/09 15:45:52 [WARN] cleanup token: 7Ki11nC8pPJJzxsGF0lAEXLs1AFSd7nKCLfK7RQ91bg
2024/09/09 15:45:54 [INFO] [dns-test.us.kg] acme: Cleaning DNS-01 challenge
2024/09/09 15:45:54 [WARN] cleanup token: t-L4putOh4Dg8wvvQcefw0V1UbjYbDeCoZj95qw25BA
2024/09/09 15:45:54 [WARN] [dns-test.us.kg] acme: cleaning up failed: huaweicloud: unknown record ID for '_acme-challenge.dns-test.us.kg.' 't-L4putOh4Dg8wvvQcefw0V1UbjYbDeCoZj95qw25BA' 
2024/09/09 15:45:54 [INFO] Skipping deactivating of valid auth: https://acme-staging-v02.api.letsencrypt.org/acme/authz-v3/13931740453
2024/09/09 15:45:55 [INFO] Deactivating auth: https://acme-staging-v02.api.letsencrypt.org/acme/authz-v3/13931740463
2024/09/09 15:45:55 Could not obtain certificates:
        error: one or more domains had a problem:
[dns-test.us.kg] [dns-test.us.kg] acme: error presenting token: huaweicloud: update record set: {"status_code":404,"request_id":"431017d6f31d98e7a041b072e8492c58","error_code":"APIGW.0101","error_message":"The API does not exist or has not been published in the environment","encoded_authorization_message":""}

ldez commented 3 weeks ago

The current error with the clean-up is not the real problem (it was expected).

The real problem is in the "present":

[dns-test.us.kg] [dns-test.us.kg] acme: error presenting token: huaweicloud: update record set: {"status_code":404,"request_id":"431017d6f31d98e7a041b072e8492c58","error_code":"APIGW.0101","error_message":"The API does not exist or has not been published in the environment","encoded_authorization_message":""}

I think it's because you don't authorize access to the update endpoint, can you check the rights related to your secret access key?

https://support.huaweicloud.com/intl/en-us/devg-apisign/api-sign-0002.html https://support.huaweicloud.com/intl/en-us/usermanual-apig/apig-ug-180530090.html

endymx commented 3 weeks ago

Fixed. New output logs:

2024/09/10 07:51:49 [INFO] [*.dns-test.us.kg, dns-test.us.kg] acme: Obtaining bundled SAN certificate
2024/09/10 07:51:51 [INFO] [*.dns-test.us.kg] AuthURL: https://acme-staging-v02.api.letsencrypt.org/acme/authz-v3/13941646363
2024/09/10 07:51:51 [INFO] [dns-test.us.kg] AuthURL: https://acme-staging-v02.api.letsencrypt.org/acme/authz-v3/13941646373
2024/09/10 07:51:51 [INFO] [*.dns-test.us.kg] acme: use dns-01 solver
2024/09/10 07:51:51 [INFO] [dns-test.us.kg] acme: Could not find solver for: tls-alpn-01
2024/09/10 07:51:51 [INFO] [dns-test.us.kg] acme: Could not find solver for: http-01
2024/09/10 07:51:51 [INFO] [dns-test.us.kg] acme: use dns-01 solver
2024/09/10 07:51:51 [INFO] [*.dns-test.us.kg] acme: Preparing to solve DNS-01
2024/09/10 07:51:53 [INFO] Wait for record set sync on dns-test.us.kg [timeout: 1m0s, interval: 2s]
2024/09/10 07:51:53 [INFO] [dns-test.us.kg] acme: Preparing to solve DNS-01
2024/09/10 07:51:54 [INFO] Wait for record set sync on dns-test.us.kg [timeout: 1m0s, interval: 2s]
2024/09/10 07:51:54 [INFO] [*.dns-test.us.kg] acme: Trying to solve DNS-01
2024/09/10 07:51:54 [INFO] [*.dns-test.us.kg] acme: Checking DNS record propagation. [nameservers=183.60.83.19:53,183.60.82.98:53]
2024/09/10 07:51:56 [INFO] Wait for propagation [timeout: 1m0s, interval: 2s]
2024/09/10 07:51:57 [INFO] [*.dns-test.us.kg] acme: Waiting for DNS record propagation.
2024/09/10 07:52:17 [INFO] [*.dns-test.us.kg] The server validated our request
2024/09/10 07:52:17 [INFO] [dns-test.us.kg] acme: Trying to solve DNS-01
2024/09/10 07:52:17 [INFO] [dns-test.us.kg] acme: Checking DNS record propagation. [nameservers=183.60.83.19:53,183.60.82.98:53]
2024/09/10 07:52:19 [INFO] Wait for propagation [timeout: 1m0s, interval: 2s]
2024/09/10 07:52:33 [INFO] [dns-test.us.kg] The server validated our request
2024/09/10 07:52:33 [INFO] [*.dns-test.us.kg] acme: Cleaning DNS-01 challenge
2024/09/10 07:52:34 [INFO] [dns-test.us.kg] acme: Cleaning DNS-01 challenge
2024/09/10 07:52:35 [WARN] [dns-test.us.kg] acme: cleaning up failed: huaweicloud: delete record: {"status_code":404,"request_id":"7153a84cdb56ea4b31fedb740e60914d","error_code":"DNS.0313","error_message":"This record set does not exist.","encoded_authorization_message":""}

2024/09/10 07:52:35 [INFO] [*.dns-test.us.kg, dns-test.us.kg] acme: Validations succeeded; requesting certificates
2024/09/10 07:52:35 [INFO] Wait for certificate [timeout: 30s, interval: 500ms]
2024/09/10 07:52:39 [INFO] [*.dns-test.us.kg] Server responded with a certificate.

ldez commented 3 weeks ago

For your information, the following error is expected because lego will try to delete 2 times the same record set.

[WARN] [dns-test.us.kg] acme: cleaning up failed: huaweicloud: delete record

As the same record set contains the 2 TXT values (because it's the same domain), and the 2 clean-ups are simultaneous, the error appears but it's just a warning, and it works as expected.

ldez commented 3 weeks ago

Now, can you provide the full log (with your domain and personal information redacted)?

I want to see the output of the rm -rf .lego and all the lines related to the execution of lego.

endymx commented 3 weeks ago

Sure, this is full logs:

root@VM-20-10-debian:~# rm -rf .lego

root@VM-20-10-debian:~# HUAWEICLOUD_ACCESS_KEY_ID=GNU*** HUAWEICLOUD_SECRET_ACCESS_KEY=xfo*** HUAWEICLOUD_REGION=cn-south-1 ./lego -m e***@gmail.com --dns huaweicloud -d *.dns-test.us.kg -d dns-test.us.kg -s https://acme-staging-v02.api.letsencrypt.org/directory run
2024/09/10 09:38:53 No key found for account e***@gmail.com. Generating a P256 key.
2024/09/10 09:38:53 Saved key to /root/.lego/accounts/acme-staging-v02.api.letsencrypt.org/e***@gmail.com/keys/e***@gmail.com.key
2024/09/10 09:38:55 Please review the TOS at https://letsencrypt.org/documents/LE-SA-v1.4-April-3-2024.pdf
Do you accept the TOS? Y/n
Y
2024/09/10 09:38:58 [INFO] acme: Registering account for e***@gmail.com
!!!! HEADS UP !!!!

Your account credentials have been saved in your Let's Encrypt
configuration directory at "/root/.lego/accounts".

You should make a secure backup of this folder now. This
configuration directory will also contain certificates and
private keys obtained from Let's Encrypt so making regular
backups of this folder is ideal.
2024/09/10 09:38:59 [INFO] [*.dns-test.us.kg, dns-test.us.kg] acme: Obtaining bundled SAN certificate
2024/09/10 09:39:00 [INFO] [*.dns-test.us.kg] AuthURL: https://acme-staging-v02.api.letsencrypt.org/acme/authz-v3/13942736503
2024/09/10 09:39:00 [INFO] [dns-test.us.kg] AuthURL: https://acme-staging-v02.api.letsencrypt.org/acme/authz-v3/13942736513
2024/09/10 09:39:00 [INFO] [*.dns-test.us.kg] acme: use dns-01 solver
2024/09/10 09:39:00 [INFO] [dns-test.us.kg] acme: Could not find solver for: tls-alpn-01
2024/09/10 09:39:00 [INFO] [dns-test.us.kg] acme: Could not find solver for: http-01
2024/09/10 09:39:00 [INFO] [dns-test.us.kg] acme: use dns-01 solver
2024/09/10 09:39:00 [INFO] [*.dns-test.us.kg] acme: Preparing to solve DNS-01
2024/09/10 09:39:02 [INFO] Wait for record set sync on dns-test.us.kg [timeout: 1m0s, interval: 2s]
2024/09/10 09:39:03 [INFO] [dns-test.us.kg] acme: Preparing to solve DNS-01
2024/09/10 09:39:03 [INFO] Wait for record set sync on dns-test.us.kg [timeout: 1m0s, interval: 2s]
2024/09/10 09:39:03 [INFO] [*.dns-test.us.kg] acme: Trying to solve DNS-01
2024/09/10 09:39:04 [INFO] [*.dns-test.us.kg] acme: Checking DNS record propagation. [nameservers=183.60.83.19:53,183.60.82.98:53]
2024/09/10 09:39:06 [INFO] Wait for propagation [timeout: 1m0s, interval: 2s]
2024/09/10 09:39:17 [INFO] [*.dns-test.us.kg] acme: Waiting for DNS record propagation.
2024/09/10 09:39:29 [INFO] [*.dns-test.us.kg] acme: Waiting for DNS record propagation.
2024/09/10 09:39:41 [INFO] [*.dns-test.us.kg] acme: Waiting for DNS record propagation.
2024/09/10 09:39:54 [INFO] [*.dns-test.us.kg] The server validated our request
2024/09/10 09:39:54 [INFO] [dns-test.us.kg] acme: Trying to solve DNS-01
2024/09/10 09:39:54 [INFO] [dns-test.us.kg] acme: Checking DNS record propagation. [nameservers=183.60.83.19:53,183.60.82.98:53]
2024/09/10 09:39:56 [INFO] Wait for propagation [timeout: 1m0s, interval: 2s]
2024/09/10 09:40:06 [INFO] [dns-test.us.kg] acme: Waiting for DNS record propagation.
2024/09/10 09:40:17 [INFO] [dns-test.us.kg] The server validated our request
2024/09/10 09:40:17 [INFO] [*.dns-test.us.kg] acme: Cleaning DNS-01 challenge
2024/09/10 09:40:17 [INFO] [dns-test.us.kg] acme: Cleaning DNS-01 challenge
2024/09/10 09:40:17 [WARN] [dns-test.us.kg] acme: cleaning up failed: huaweicloud: delete record: {"status_code":404,"request_id":"152167c7886a08961fe83f516e531f38","error_code":"DNS.0313","error_message":"This record set does not exist.","encoded_authorization_message":""}

2024/09/10 09:40:17 [INFO] [*.dns-test.us.kg, dns-test.us.kg] acme: Validations succeeded; requesting certificates
2024/09/10 09:40:18 [INFO] Wait for certificate [timeout: 30s, interval: 500ms]
2024/09/10 09:40:19 [INFO] [*.dns-test.us.kg] Server responded with a certificate.
root@VM-20-10-debian:~#

ldez commented 3 weeks ago

Thank you, everything seems OK now.

go-acme / lego

Add DNS provider for HuaweiCloud #2267