Streamline custom domain support when using PublishAsAzureContainerApp

[mitchdenny]

Developers deploying a .NET Aspire application to Azure via AZD will often want to associate the container app resource with a custom domain name. This means that they need to create the necessary customizations on the ingress settings on the container app resource, but also set up a certificate in the in the managed environment.

This is a bit of a chicken and egg problem because in order to complete this process the DNS records need to exist which verify ownership and intent to service traffic via container apps. Specifically, there is a TXT record that is the hash of the resource owner's subscription ID and a seed value in addition to a CNAME record that points to *.trafficmanager.net (you can use a more specific endpoint but in this scenario you would use the traffic manager endpoint because you can't know the randomly assigned FQDN up front).

The code to configure the Bicep generation via Azure Provisioning looks something like this (still testing):

       .PublishAsAzureContainerApp((module, app) =>
       {
           var environment = ContainerAppManagedEnvironment.FromExisting("environment");
           environment.Name = app.EnvironmentId;
           module.Add(environment);

           var managedCert = new ContainerAppManagedCertificate("cert");
           managedCert.Parent = environment;
           managedCert.Properties = new ManagedCertificateProperties()
           {
               SubjectName = "mydomain.mycompany.com"
           };
           module.Add(managedCert);

           app.Configuration.Value!.Ingress!.Value!.CustomDomains = new Azure.Provisioning.BicepList<ContainerAppCustomDomain>()
           {
                new ContainerAppCustomDomain()
                {
                    BindingType = ContainerAppCustomDomainBindingType.SniEnabled,
                    CertificateId = managedCert.Id,
                    Name = "mydomain.mycompany.com"
                }
           };

           // Scale to 0
           app.Template.Value!.Scale.Value!.MinReplicas = 0;
       });

The code to generate the content for the TXT record is here:

using System.Security.Cryptography;
using System.Text;

var hash = SHA256.Create();
string uniqueId = "<subscription id>" + "282EF";
var hashed = hash.ComputeHash(Encoding.UTF8.GetBytes(uniqueId));

var sb = new StringBuilder();

foreach (var b in hashed)
{
    sb.Append(b.ToString("X2"));
}

Console.WriteLine(sb.ToString());

[mitchdenny]

OK so getting the cert created and associated with the custom domain on the container app seems to be a bust for now given a bit of a cycle problem that would require two deployments. Rather than go down that rabbit hole I've explored getting us back to where we were with .NET Aspire 8.0 where we had the flag in azd to preserve domains.

Here is what you AppHost Program.cs file could look like (taken from playground where I am experimenting right now:

// Licensed to the .NET Foundation under one or more agreements.
// The .NET Foundation licenses this file to you under the MIT license.
using Azure.Provisioning.AppContainers;
using Azure.Provisioning.Expressions;

var builder = DistributedApplication.CreateBuilder(args);

var customDomain = builder.AddParameter("customDomain");
var certificatId = builder.AddParameter("certificateId");

// Testing secret parameters
var param = builder.AddParameter("secretparam", "fakeSecret", secret: true);

// Testing volumes
var redis = builder.AddRedis("cache")
    .WithLifetime(ContainerLifetime.Persistent)
    .WithDataVolume();

// Testing secret outputs
var cosmosDb = builder.AddAzureCosmosDB("account")
                      .RunAsEmulator(c => c.WithLifetime(ContainerLifetime.Persistent))
                      .AddDatabase("db");

// Testing a connection string
var blobs = builder.AddAzureStorage("storage")
                   .RunAsEmulator(c => c.WithLifetime(ContainerLifetime.Persistent))
                   .AddBlobs("blobs");

builder.AddProject<Projects.AzureContainerApps_ApiService>("api")
       .WithExternalHttpEndpoints()
       .WithReference(blobs)
       .WithReference(redis)
       .WithReference(cosmosDb)
       .WithEnvironment("VALUE", param)
       .PublishAsAzureContainerApp((module, app) =>
       {
           var certificatIdParameter = certificatId.AsProvisioningParameter(module);
           var customDomainParameter = customDomain.AsProvisioningParameter(module);

           var bindingTypeConditional = new ConditionalExpression(
               new BinaryExpression(
                   new IdentifierExpression(certificatIdParameter.IdentifierName),
                   BinaryOperator.NotEqual,
                   new StringLiteral(string.Empty)),
               new StringLiteral("SniEnabled"),
               new StringLiteral("Disabled")
               );

           var certificateOrEmpty = new ConditionalExpression(
               new BinaryExpression(
                   new IdentifierExpression(certificatIdParameter.IdentifierName),
                   BinaryOperator.NotEqual,
                   new StringLiteral(string.Empty)),
               new IdentifierExpression(certificatIdParameter.IdentifierName),
               new NullLiteral()
               );

           app.Configuration.Value!.Ingress!.Value!.CustomDomains = new Azure.Provisioning.BicepList<ContainerAppCustomDomain>()
           {

                new ContainerAppCustomDomain()
                {
                    BindingType = bindingTypeConditional,
                    Name = "mydomain.mitchdenny.dev",
                    CertificateId = certificateOrEmpty
                }
           };

           // Scale to 0
           app.Template.Value!.Scale.Value!.MinReplicas = 0;
       });

#if !SKIP_DASHBOARD_REFERENCE
// This project is only added in playground projects to support development/debugging
// of the dashboard. It is not required in end developer code. Comment out this code
// or build with `/p:SkipDashboardReference=true`, to test end developer
// dashboard launch experience, Refer to Directory.Build.props for the path to
// the dashboard binary (defaults to the Aspire.Dashboard bin output in the
// artifacts dir).
builder.AddProject<Projects.Aspire_Dashboard>(KnownResourceNames.AspireDashboard);
#endif

builder.Build().Run();

Here is the workflow:

1. Deploy the AppHost but do not provide the certificateId and customDomain parameter values (just hit ENTER).
2. The container app will be deployed and be available on its randomly assigned hostname.
3. Go into the container app and set up a custom domain and wait for it to be configured.
4. Note the ID of the certificate resource. It should be in the form: /subscriptions/<certificate id>/resourceGroups/<resource group id>/providers/Microsoft.App/managedEnvironments/<environment name>/managedCertificates/<cert id ... visible in portal>
5. Update the config.json file with these values (or however you pass them into AZD).
6. Deploy again ... the custom domain should be preserved.

[mitchdenny]

Revised so only the cert name and domain is neeed (not the whole cert ID):

// Licensed to the .NET Foundation under one or more agreements.
// The .NET Foundation licenses this file to you under the MIT license.
using Azure.Provisioning;
using Azure.Provisioning.AppContainers;
using Azure.Provisioning.Expressions;

var builder = DistributedApplication.CreateBuilder(args);

var customDomain = builder.AddParameter("customDomain");
var certificateName = builder.AddParameter("certificateName");

// Testing secret parameters
var param = builder.AddParameter("secretparam", "fakeSecret", secret: true);

// Testing volumes
var redis = builder.AddRedis("cache")
    .WithLifetime(ContainerLifetime.Persistent)
    .WithDataVolume();

// Testing secret outputs
var cosmosDb = builder.AddAzureCosmosDB("account")
                      .RunAsEmulator(c => c.WithLifetime(ContainerLifetime.Persistent))
                      .AddDatabase("db");

// Testing a connection string
var blobs = builder.AddAzureStorage("storage")
                   .RunAsEmulator(c => c.WithLifetime(ContainerLifetime.Persistent))
                   .AddBlobs("blobs");

builder.AddProject<Projects.AzureContainerApps_ApiService>("api")
       .WithExternalHttpEndpoints()
       .WithReference(blobs)
       .WithReference(redis)
       .WithReference(cosmosDb)
       .WithEnvironment("VALUE", param)
       .PublishAsAzureContainerApp((module, app) =>
       {
           var containerAppManagedEnvironmentIdParameter = module.GetResources().OfType<ProvisioningParameter>().Single(
                p => p.IdentifierName == "outputs_azure_container_apps_environment_id");
           var certificatNameParameter = certificateName.AsProvisioningParameter(module);
           var customDomainParameter = customDomain.AsProvisioningParameter(module);

           var bindingTypeConditional = new ConditionalExpression(
               new BinaryExpression(
                   new IdentifierExpression(certificatNameParameter.IdentifierName),
                   BinaryOperator.NotEqual,
                   new StringLiteral(string.Empty)),
               new StringLiteral("SniEnabled"),
               new StringLiteral("Disabled")
               );

           var certificateOrEmpty = new ConditionalExpression(
               new BinaryExpression(
                   new IdentifierExpression(certificatNameParameter.IdentifierName),
                   BinaryOperator.NotEqual,
                   new StringLiteral(string.Empty)),
               new InterpolatedString(
                   "{0}/managedCertificates/{1}",
                   [
                    new IdentifierExpression(containerAppManagedEnvironmentIdParameter.IdentifierName),
                    new IdentifierExpression(certificatNameParameter.IdentifierName)
                    ]),
               new NullLiteral()
               );

           app.Configuration.Value!.Ingress!.Value!.CustomDomains = new Azure.Provisioning.BicepList<ContainerAppCustomDomain>()
           {
                new ContainerAppCustomDomain()
                {
                    BindingType = bindingTypeConditional,
                    Name = new IdentifierExpression(customDomainParameter.IdentifierName),
                    CertificateId = certificateOrEmpty
                }
           };

           // Scale to 0
           app.Template.Value!.Scale.Value!.MinReplicas = 0;
       });

#if !SKIP_DASHBOARD_REFERENCE
// This project is only added in playground projects to support development/debugging
// of the dashboard. It is not required in end developer code. Comment out this code
// or build with `/p:SkipDashboardReference=true`, to test end developer
// dashboard launch experience, Refer to Directory.Build.props for the path to
// the dashboard binary (defaults to the Aspire.Dashboard bin output in the
// artifacts dir).
builder.AddProject<Projects.Aspire_Dashboard>(KnownResourceNames.AspireDashboard);
#endif

builder.Build().Run();

[csharpfritz]

I like where you're going with this...

[Steinblock]

While the solution to use PublishAsAzureContainerApp is very powerful, I'd like to see a more generic solution

Like WithExternalHttpEndpoints() sets endpoint.IsExternal to true, which, in context of an azure container app, instructs the ingress to change Limited to container apps Environment to Accepting traffic from anywhere

I could image an overload that takes a list of domains

var project = builder.AddProject<Projects.Backedbd>("backend")
    .WithExternalHttpEndpoints(["myapp.mydomain.tld"])
    .WithReference(postgresdb)

or another extension method.

var project = builder.AddProject<Projects.Backedbd>("backend")
    .WithExternalHttpEndpoints()
    .WithCustomDomains(["myapp.mydomain.tld"]);
    .WithReference(postgresdb)

Even if I'm deploying to Azure container apps myself, from my point of view the Aspire AppHost > Program.cs should provide a level of abstraction that does not need to define container app specific methods for such common things like certificate creation.

[davidfowl]

Related https://github.com/dotnet/aspire/issues/1498