search

shibayan / keyvault-acmebot

Automated ACME SSL/TLS certificates issuer for Azure Key Vault (App Service / Container Apps / App Gateway / Front Door / CDN / others)
Apache License 2.0
895 stars 233 forks source link

Add multiple domains from the command-line #230

Closed soderlind closed 3 years ago

soderlind commented 3 years ago

Is your feature request related to a problem? Please describe. I need to add multiple domains (50+)

Describe the solution you'd like I would like to add the domains using az cli, eg: az keyvault-acmebot --resource-group $RG --domain-name mydomain.tld ..etc

Describe alternatives you've considered Add one by one using the web UI.

shibayan commented 3 years ago

Command line implementation may be difficult due to authentication issues. For starters, what do you think about adding something like an advanced mode to the web UI that would allow you to add all the domains you need at once?

The idea I was thinking of was to implement it as a Custom Provider in Azure Resource Manager, which when integrated with ARM would be available from tools like Azure CLI (although Custom Provider is in preview).

soderlind commented 3 years ago

A way of adding multiple domains via a textarea, one domain per line, would help a lot :)

And if you create a custom provider, I could do something like:

az resource invoke-action --action {actionName} \
                          --ids /subscriptions/{subscriptionId}/resourceGroups/{resourceGroupName}/providers/Microsoft.CustomProviders/resourceProviders/{resourceProviderName} \
                          --request-body \
                            '{
                                "domainNames": [ "mydomain1.tld","mydomain2.tld","mydomain3.tld"]
                            }'
shibayan commented 3 years ago

I'm going to add an advanced mode (which can be specified in the text area) to the Web UI as soon as possible. (That will be addressed in this issue)

The implementation as a Custom Provider will be available not only from Azure CLI, but also from ARM Template and Terraform, which is obviously useful.

I'm worried about the roadmap for Custom Provider itself, but it's worth implementing. I will create another issue for it.

shibayan commented 3 years ago

@soderlind Although it is a very simple web UI, you can create SANs certificates by specifying one DNS name per line in the text area. Web UI Endpoint: https://your-acmebot-name.azurewebsites.net/bulk-certificate

Note that in Let's Encrypt the maximum number of domains that can be specified in a SANs certificate is 100. You should also be aware of the rate limit. https://letsencrypt.org/docs/rate-limits/

soderlind commented 3 years ago

@shibayan Thank you very much!

Updating keyvault-acmebot is just installing it on top of existing installation?

shibayan commented 3 years ago

@soderlind Acmebot will be updated automatically, and you will need to restart it in Azure Portal. That's all.

soderlind commented 3 years ago

@shibayan, bulk adding domains gives me(tried with just a few):

  Orchestrator function 'IssueCertificate' failed: The activity function 'CheckDnsChallenge' failed: "_acme- 
  challenge.opengovernment.teststeder.regjeringen.no did not resolve.". See the function execution logs for additional details.

opengovernment.teststeder.regjeringen.no points to our front door.

Call stack:

Microsoft.Azure.WebJobs.Host.FunctionInvocationException:
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw (System.Private.CoreLib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=abcdefgh)
   at Microsoft.Azure.WebJobs.Host.Executors.FunctionExecutor+<ExecuteWithLoggingAsync>d__20.MoveNext (Microsoft.Azure.WebJobs.Host, Version=3.0.25.0, Culture=neutral, PublicKeyToken=123456789Microsoft.Azure.WebJobs.Host, Version=3.0.25.0, Culture=neutral, PublicKeyToken=123456789: C:\projects\azure-webjobs-sdk-rqm4t\src\Microsoft.Azure.WebJobs.Host\Executors\FunctionExecutor.csMicrosoft.Azure.WebJobs.Host, Version=3.0.25.0, Culture=neutral, PublicKeyToken=123456789: 330)
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw (System.Private.CoreLib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=abcdefgh)
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess (System.Private.CoreLib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=abcdefgh)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification (System.Private.CoreLib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=abcdefgh)
   at System.Runtime.CompilerServices.TaskAwaiter`1.GetResult (System.Private.CoreLib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=abcdefgh)
   at Microsoft.Azure.WebJobs.Host.Executors.FunctionExecutor+<TryExecuteAsync>d__15.MoveNext (Microsoft.Azure.WebJobs.Host, Version=3.0.25.0, Culture=neutral, PublicKeyToken=123456789Microsoft.Azure.WebJobs.Host, Version=3.0.25.0, Culture=neutral, PublicKeyToken=123456789: C:\projects\azure-webjobs-sdk-rqm4t\src\Microsoft.Azure.WebJobs.Host\Executors\FunctionExecutor.csMicrosoft.Azure.WebJobs.Host, Version=3.0.25.0, Culture=neutral, PublicKeyToken=123456789: 94)
Inner exception KeyVault.Acmebot.RetriableActivityException handled at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw:
   at KeyVault.Acmebot.Functions.SharedActivity+<CheckDnsChallenge>d__15.MoveNext (KeyVault.Acmebot, Version=3.4.2.0, Culture=neutral, PublicKeyToken=nullKeyVault.Acmebot, Version=3.4.2.0, Culture=neutral, PublicKeyToken=null: /home/runner/work/keyvault-acmebot/keyvault-acmebot/KeyVault.Acmebot/Functions/SharedActivity.csKeyVault.Acmebot, Version=3.4.2.0, Culture=neutral, PublicKeyToken=null: 202)
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw (System.Private.CoreLib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=abcdefgh)
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess (System.Private.CoreLib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=abcdefgh)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification (System.Private.CoreLib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=abcdefgh)
   at System.Runtime.CompilerServices.TaskAwaiter.GetResult (System.Private.CoreLib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=abcdefgh)
   at Microsoft.Azure.WebJobs.Host.Executors.VoidTaskMethodInvoker`2+<InvokeAsync>d__2.MoveNext (Microsoft.Azure.WebJobs.Host, Version=3.0.25.0, Culture=neutral, PublicKeyToken=123456789Microsoft.Azure.WebJobs.Host, Version=3.0.25.0, Culture=neutral, PublicKeyToken=123456789: C:\projects\azure-webjobs-sdk-rqm4t\src\Microsoft.Azure.WebJobs.Host\Executors\VoidTaskMethodInvoker.csMicrosoft.Azure.WebJobs.Host, Version=3.0.25.0, Culture=neutral, PublicKeyToken=123456789: 20)
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw (System.Private.CoreLib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=abcdefgh)
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess (System.Private.CoreLib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=abcdefgh)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification (System.Private.CoreLib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=abcdefgh)
   at System.Runtime.CompilerServices.TaskAwaiter`1.GetResult (System.Private.CoreLib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=abcdefgh)
   at Microsoft.Azure.WebJobs.Host.Executors.FunctionInvoker`2+<InvokeAsync>d__10.MoveNext (Microsoft.Azure.WebJobs.Host, Version=3.0.25.0, Culture=neutral, PublicKeyToken=123456789Microsoft.Azure.WebJobs.Host, Version=3.0.25.0, Culture=neutral, PublicKeyToken=123456789: C:\projects\azure-webjobs-sdk-rqm4t\src\Microsoft.Azure.WebJobs.Host\Executors\FunctionInvoker.csMicrosoft.Azure.WebJobs.Host, Version=3.0.25.0, Culture=neutral, PublicKeyToken=123456789: 52)
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw (System.Private.CoreLib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=abcdefgh)
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess (System.Private.CoreLib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=abcdefgh)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification (System.Private.CoreLib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=abcdefgh)
   at System.Runtime.CompilerServices.TaskAwaiter`1.GetResult (System.Private.CoreLib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=abcdefgh)
   at Microsoft.Azure.WebJobs.Host.Executors.FunctionExecutor+<InvokeWithTimeoutAsync>d__27.MoveNext (Microsoft.Azure.WebJobs.Host, Version=3.0.25.0, Culture=neutral, PublicKeyToken=123456789Microsoft.Azure.WebJobs.Host, Version=3.0.25.0, Culture=neutral, PublicKeyToken=123456789: C:\projects\azure-webjobs-sdk-rqm4t\src\Microsoft.Azure.WebJobs.Host\Executors\FunctionExecutor.csMicrosoft.Azure.WebJobs.Host, Version=3.0.25.0, Culture=neutral, PublicKeyToken=123456789: 559)
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw (System.Private.CoreLib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=abcdefgh)
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess (System.Private.CoreLib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=abcdefgh)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification (System.Private.CoreLib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=abcdefgh)
   at System.Runtime.CompilerServices.TaskAwaiter`1.GetResult (System.Private.CoreLib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=abcdefgh)
   at Microsoft.Azure.WebJobs.Host.Executors.FunctionExecutor+<ExecuteWithWatchersAsync>d__26.MoveNext (Microsoft.Azure.WebJobs.Host, Version=3.0.25.0, Culture=neutral, PublicKeyToken=123456789Microsoft.Azure.WebJobs.Host, Version=3.0.25.0, Culture=neutral, PublicKeyToken=123456789: C:\projects\azure-webjobs-sdk-rqm4t\src\Microsoft.Azure.WebJobs.Host\Executors\FunctionExecutor.csMicrosoft.Azure.WebJobs.Host, Version=3.0.25.0, Culture=neutral, PublicKeyToken=123456789: 505)
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw (System.Private.CoreLib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=abcdefgh)
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess (System.Private.CoreLib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=abcdefgh)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification (System.Private.CoreLib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=abcdefgh)
   at System.Runtime.CompilerServices.TaskAwaiter.GetResult (System.Private.CoreLib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=abcdefgh)
   at Microsoft.Azure.WebJobs.Host.Executors.FunctionExecutor+<ExecuteWithLoggingAsync>d__20.MoveNext (Microsoft.Azure.WebJobs.Host, Version=3.0.25.0, Culture=neutral, PublicKeyToken=123456789Microsoft.Azure.WebJobs.Host, Version=3.0.25.0, Culture=neutral, PublicKeyToken=123456789: C:\projects\azure-webjobs-sdk-rqm4t\src\Microsoft.Azure.WebJobs.Host\Executors\FunctionExecutor.csMicrosoft.Azure.WebJobs.Host, Version=3.0.25.0, Culture=neutral, PublicKeyToken=123456789: 283)
shibayan commented 3 years ago

@soderlind Cannot seem to resolve certain validation DNS records. Are there any _acme-challenge records added to Azure DNS that correspond to this domain? _acme-challenge.opengovernment.teststeder.regjeringen.no.

I have checked and it is not resolving correctly, so it may not be registering to Azure DNS. https://digwebinterface.com/?hostnames=_acme-challenge.opengovernment.teststeder.regjeringen.no&type=TXT&ns=resolver&useresolver=8.8.4.4&nameservers=

It looks like a glitch in Acmebot, so I'll check it out. Are you using Azure DNS Child zone? I need some information around DNS.

soderlind commented 3 years ago

Might be my bad. Azure DNS is in another subscription.

www is in a child zone, but all my testsites (NN.teststeder.regjeringen.no) are not, and they are CNAMEd to our testing front door.

shibayan commented 3 years ago

Do you have an Azure DNS with the same name?

It is possible that the Azure DNS that exists for the subscription ID configured in Acmebot is not actually being used.

soderlind commented 3 years ago

Tested with Azure DNS in same subscriptions as your Key Vault Acmebot, and everything works fine.

soderlind commented 3 years ago

When I use bulk add, do you create one SAN certificate per bulk job (ref max domains = 100 pr SAN certs.) ?

In a previous tool we did (PHP based), we grouped the domains and spread the over multiple SAN certs, with max 5 domains per cert. It made updating certs faster (ref #235)

shibayan commented 3 years ago

Yes, the specified domain list will be set to SANs for a single certificate.

I've been looking into performance, but it seems to take a long time to verify Let's Encrypt ownership in the first place, so it seemed difficult to improve.

shibayan commented 3 years ago

Resolved?

soderlind commented 3 years ago

Yes, using azure dns in same subscription is ok by me.

soderlind commented 3 years ago

@shibayan Tried to use /bulk-certificate today and got this error:

Orchestrator function 'IssueCertificate' failed: The activity function 'FinalizeOrder' failed: "Value cannot be null. (Parameter 'value')". See the function execution logs for additional details.

/add-certificate works fine.

soderlind commented 3 years ago

@shibayan Saw that I can add multiple domains using /add-certificate so I don't think I need /bulk-certificate

shibayan commented 3 years ago

Check the Web UI as it may not be able to keep up with API specification changes.