Unable to have dependencies between bicep and yaml based resources

[davidfowl]

Code is here https://github.com/davidfowl/EventGridDemo

Follow up from https://github.com/Azure/azure-dev/issues/3839

var builder = DistributedApplication.CreateBuilder(args);

var api = builder.AddProject<Projects.EventGridDemo_Api>("api")
                     .WithExternalHttpEndpoints();

var eventGrid = builder.AddEventGridWebHook("grid")
                       .WithWebhookUrl($"{api.GetEndpoint("https")}/hook");

builder.AddProject<Projects.EventGridDemo_Publisher>("publisher")
       .WithReference(eventGrid);

builder.Build().Run();

In this sample, the webhook url is properly expressed in the bicep:

module grid 'grid/eventgridwebhook.bicep' = {
  name: 'grid'
  scope: rg
  params: {
    location: location
    principalId: resources.outputs.MANAGED_IDENTITY_PRINCIPAL_ID
    principalType: 'ServicePrincipal'
    topicName: 'grid'
    webHookEndpoint: 'https://api.${resources.outputs.AZURE_CONTAINER_APPS_ENVIRONMENT_DEFAULT_DOMAIN}/hook'
  }
}

The problem is that address needs to be exposed before the event grid subscription can be deployed. The ARM deployment fails with:

ERROR: error executing step command 'provision': deployment failed: error deploying infrastructure: deploying to subscription:

Deployment Error Details:
Url validation: Webhook validation handshake failed for https://api.<redacted>/hook. Http POST request failed with response code Unknown. For troubleshooting, visit https://aka.ms/esvalidation.

This bring into question the entire "provision" vs "deploy" model. In theory I should be able to make this work by adding a dependency between the container app and the event grid resource, but that isn't possible right now (even manually) because of the deploy vs provision split.

@mitchdenny and I have discussed this end to end where we have change the overall flow to be:

Publish container images

1. Get or Create container registry
2. Publish projects and docker files

Provision infrastructure

3. This is where you create all of the resources, including the container apps. This would allow us to make everything bicep and to set dependencies between container apps and other azure resources.

It should still be possible to deploy applications individually without provisioning. That's where this gets a bit wonky and requires a bit more ideation.

[davidfowl]

After some discussion with @vhvb1989 I ended up with a bootstrapped container image and a bootstrap container app that can validate the webhook.

I still needed to do azd infra synth so I could add the bootstrap container app before the event grid resource.

The feature ask here might be to add a known parameter which is the container app environment id. That would allow me to build this as a separate bicep based resource in the app model.

[weikanglim]

Longer-term ask related to #2650, #813

[mitchdenny]

Wouldn't you still want to be able to control some things about the container app itself?

[davidfowl]

@vhvb1989 gonna leave this open

[ellismg]

Instead of moving the order in which azd does things around like this, I wish there was a way for us to produce a single artifact that represented the entire end to end deployment, and have azd interpret this.

Thinking about Aspire today, there's a few things we can't easily do in ARM:

1. We can't easily build and push container images to a registry as part of an ARM deployment.
2. For bind mounts we can't easily upload artifacts that blob storage as part of an ARM deployment.

There's sort of the escape hatch in arm to use deployment scripts to run custom actions, but the problem leveraging this that we need some place to stage files from the user's machine into a place where the deployment scripts could access them. In think in practice this would mean creating a blob storage account, a MI to represent the principal that runs the script, and then grant the MI access to the blob storage account via azure RBAC.

Part of me wonders if we should be sketching out what a bicep file that solved this problem would look like, one that would represent the complete deployment, with some custom resources that azd understood. For example:

// deployAll.bicep

module sharedInfra './resources.bicep' = {
  name: 'sharedInfra'
  params: {
    // Threading params here...
  }
}

resource appContainer "Azd/containerImage@2024-05-24-preview" {
   sourceProject: '/path/to/App.csproj'
   targetImage: '{sharedInfra.outputs.AZURE_CONTAINER_REGISTRY}/my-aspire-project/app-dev:azd-1716575840'
}

module appInfra './services/app.bicep'
  name: 'appInfra'
  params: {
     imageId: appContainer.outputs.imageId
     // Threading params here...
  }
}

While we could not deploy this directly via ARM, azd could break it into parts based on the dependency graph and sequence the individual operations. First it would do an ARM Deployment of that module, then it would do the build and push of the container image and then run the deployment for the app.bicep.

It feels like this would give us a complete view of the overall deployment, and azd would just be responsible for driving it to ground, working with ARM to most of the heavy lifting (you could image this top level main.bicep only supports modules references and a few resouce types under the "AZD RP".

I imagine we could play similar tricks for resources which represented the push to bind mounts.

Wondering if thinking about things at this level may be helpful?

[davidfowl]

Yes I like this line of thinking!

This is my branch that generates a manifest with the complete set of bicep

https://github.com/dotnet/aspire/blob/7a140d37d79b9146efab17a71148c22d54dffce3/playground/mongo/Mongo.AppHost/app.module.bicep https://github.com/dotnet/aspire/blob/7a140d37d79b9146efab17a71148c22d54dffce3/playground/mongo/Mongo.AppHost/aspire-manifest.json

You can see this inputs object that has the container images that were pushed on them