Add support for Pulumi - Githubissues

savannahostrowski commented 2 years ago

From DevBlogs:

Please upvote if this is important to you!

jaxxstorm commented 2 years ago

Pulumi currently supports a YAML authoring language, which is convertible to a fully fledged programming language.

Being able to output Pulumi YAML, then running pulumi convert would be awesome

Trubador commented 2 years ago

Why do you think the code that pulumi creates with its cli is not sufficient? I am just curious.

TechWatching commented 2 years ago

Why do you think the code that pulumi creates with its cli is not sufficient? I am just curious.

It's not that pulumi cli is not sufficient but about being able to use azd with pulumi. Azure Developer CLI is not only about Infrastructure as Code but also about generating the application code, generating the pipelines to deploy the code, deploying the application, and monitoring the application, ... So for someone who wants to manage its application using azd (code, build, deploy, monitor) but with the infrastructure code using Pulumi, support for Pulumi in azd is a must have. Or maybe I misunderstood the question and your question was about what @jaxxstorm said?

Trubador commented 2 years ago

Why do you think the code that pulumi creates with its cli is not sufficient? I am just curious.

It's not that pulumi cli is not sufficient but about being able to use azd with pulumi. Azure Developer CLI is not only about Infrastructure as Code but also about generating the application code, generating the pipelines to deploy the code, deploying the application, and monitoring the application, ... So for someone who wants to manage its application using azd (code, build, deploy, monitor) but with the infrastructure code using Pulumi, support for Pulumi in azd is a must have. Or maybe I misunderstood the question and your question was about what @jaxxstorm said?

I can see the benefit in that if everything is generated automatically with 1 command thus saving time. It was not related to @jaxxstorm comment.

TechWatching commented 2 years ago

Why do you think the code that pulumi creates with its cli is not sufficient? I am just curious.

It's not that pulumi cli is not sufficient but about being able to use azd with pulumi. Azure Developer CLI is not only about Infrastructure as Code but also about generating the application code, generating the pipelines to deploy the code, deploying the application, and monitoring the application, ... So for someone who wants to manage its application using azd (code, build, deploy, monitor) but with the infrastructure code using Pulumi, support for Pulumi in azd is a must have. Or maybe I misunderstood the question and your question was about what @jaxxstorm said?

I can see the benefit in that if everything is generated automatically with 1 command thus saving time. It was not related to @jaxxstorm comment.

Yes, another related benefit is to be able to share templates that contain the application and the infrastructure code. Pulumi CLI is only about the infrastructure code. For people getting started using Azure, I believe it's nice to have some boilerplate (infra+app) to build an application that can be directly deployed to Azure.

Trubador commented 2 years ago

Why do you think the code that pulumi creates with its cli is not sufficient? I am just curious.

It's not that pulumi cli is not sufficient but about being able to use azd with pulumi. Azure Developer CLI is not only about Infrastructure as Code but also about generating the application code, generating the pipelines to deploy the code, deploying the application, and monitoring the application, ... So for someone who wants to manage its application using azd (code, build, deploy, monitor) but with the infrastructure code using Pulumi, support for Pulumi in azd is a must have. Or maybe I misunderstood the question and your question was about what @jaxxstorm said?

I can see the benefit in that if everything is generated automatically with 1 command thus saving time. It was not related to @jaxxstorm comment.

Yes, another related benefit is to be able to share templates that contain the application and the infrastructure code. Pulumi CLI is only about the infrastructure code. For people getting started using Azure, I believe it's nice to have some boilerplate (infra+app) to build an application that can be directly deployed to Azure.

Yes defintely. I was actually working on building a SaaS template for azure some time ago so I can see the benefit of that.

VincentH-Net commented 7 months ago

@savannahostrowski does roadmap mean this is planned? Any details? It would be great if devs could use .NET Aspire, which enables automated deployment via AZD, together with Pulumi. I'm working on a project where I'd like to do that.

So far I have not seen concrete advice on how to combine / plans for supporting that in the bits - see discussion on pulumi repo

ellismg commented 7 months ago

It would be great if devs could use .NET Aspire, which enables automated deployment via AZD, together with Pulumi. I'm working on a project where I'd like to do that.

I'd like to better understand the scenario here - specifically what "together with Pulumi" ends up meaning. Do you simply want azd to use pulumi to deploy the infrastructure for your Aspire app behind the scenes? Are you trying to combine the infrastructure from the Aspire application with infrastructure managed directly by a Pulumi program?

I suspect for the former - just wanting to use pulumi to do the deployment of the infrastructure that your Aspire application needs - so you get like a Pulumi stack you can see in the console, can leverage esc or other stuff - is something we'd be able to support (there is the interesting issue that you point to about how today we generate bicep and for Pulumi we'd need to either generate something else or use a converter to convert the bicep to something Pulumi understands).

To be frank, the second case is harder - we currently don't have a great story for using generated infrastructure from Aspire together with hand authored infrastructure for other resources - we'll need to come up with a solution for this independent of support for Pulumi (since azd wants to support a polyglot environment) but we haven't thought about the design in great detail yet.

VincentH-Net commented 7 months ago

I'd like to better understand the scenario here

Thanks for asking @ellismg. I don't have Pulumi experience yet so I appreciate any insights / advice from those who do - it's fortunate that you have worked there so I'm interested in your thoughts.

My main goal is to simplify Azure distributed application development for C# developers, and to enable them to use their core C# skills + tools in as many areas as possible.

Aspire serves that goal very well (especially when combined with MS Orleans). However while with the AZD support Aspire extends into Azure IaC to a degree, Aspire is not a full fledged IaC solution.
Pulumi also fits very well, and is a mature and complete IaC solution with (amongst others) C# support for Azure.

What I imagine AZD Pulumi support to enable:

From an Aspire user perspective, you could just supply additional parameter(s) to the azd up command to make that infrastructure a seamless part of what Pulumi creates and knows about (so Pulumi tools can be used for that stack).
A common case is that some Azure resource type is not (yet) supported in Aspire but it is a seamless part of the application infrastructure (e.g. an Azure SignalR resource). I would like to be able to create that resource in a Pulumi program, and use e.g. a connection string for that resource in the application.

Would the above be possible with the first approach you described?

for azd to use pulumi to deploy the infrastructure for your Aspire app

(I suspect so because no direct Aspire - Pulumi integration is needed; the application code just uses resources defined by both)

Another case (not encountered yet but I only did Aspire POC's so far) is that you need more control of the Aspire defined resource than Aspire enables, so you would want to modify a resource defined by Aspire. I suspect that would require the second approach you described? A fallback would be to just make a resource definition either fully owned by Pulumi or by Aspire, not both.

In summary, if my reasoning above is correct the first approach you described should be sufficient, while the second polyglot approach seems a more powerful option further down the road.

Does this make sense from your perspective? If you see limitations or other options I'm also interested - thanks!

ellismg commented 7 months ago

@VincentH-Net

These are really great questions and really get to the heart of some of the problems we're trying to solve as continue towards a GA of Aspire and improve the azd support for it.

As you mentioned - I worked at Pulumi for a while and have a pretty good idea about what a story in Pulumi might look like (granted the team over there has done a ton of amazing work since I have left, but I feel like I still have a pretty good handle on the model). I think the answers to these questions depends on if you are looking at the problem through a "pulumi first" lens or an "azd first" lens. Since I work on azd now - I'll try to answer these questions here from the azd viewpoint, but at some point I will also try to go to the Pulomi discussion and add my thoughts as a guy who had his Pulumi hat on instead of his azd hat.

However while with the AZD support Aspire extends into Azure IaC to a degree, Aspire is not a full fledged IaC solution.

I think this is a bit of a philosophical question - my general view is that Aspire is/can be this but there are just some gaps today in the end to end that we need to go improve. You've outlined some of them:

The Aspire.Hosting.Azure package provides a set of Aspire components for working with Azure resources - but there is not complete coverage for the Azure platform. Many popular services do not have .AddXYZ() methods to be used on the DistributedApplicationBuilder type. Because of this - you can't use Aspire today to describe the complete shape of your distributed application if you need these services.
For the types that are in Aspire.Hosting.Azure today there are not a complete set of knobs to control every setting on the service you want to control. For example, there's a component that allows you to create an Azure Blob Storage Account but there are not knobs you would need to configure properties (for example, imagine you were trying to use it to store and host the assets for a static website - the component does not give you everything you need to control say the custom 404 page, leading you to need to use the az CLI or the Portal or something to control these knobs after the application has been deployed.
The story for "instead of spinning up this resource, I want you to connect to an existing one that has already been provisioned by someone" isn't smooth yet. This means even if you use some other IaC program like terraform or pulumi (or even just write straight bicep or arm templates and us az to deploy them) using these resources from your Aspire app can be challenging. Also if you want to do something like, in the Development environment I provision an database but in the Production environment I connect to an existing database and have a great end to end experience things are really rough today.

We've been talking about how we support all these cases.

For (1) we're looking at how we can auto-generate these Aspire components in the same way that the Azure SDK team generates much of the code for the .NET Azure SDKs. We're currently in the ideation stage of this work stream, but we have some really good ideas and I suspect over the next few previews you'll start seeing stuff come online here. We're also looking at the ability to do something like builder.AddAzureDeployment("mycustomresource", <path-to-a-bicep-or-arm-template>) which will allow you to insert arbitrary Bicep or ARM templates into your Aspire application, which azd will deploy with the rest of the infrastructure of your application. I expect that this primitive is the thing that all the auto-generated code will build upon. My guess is that starting around preview 4 you'll see support for this primitive.

For (2) We're currently thinking about a strategy that allows you to add annotations to resources in your app host and tools like azd can interpret these annotations to control certain parts of the resource. For example, today when you do builder.AddPostgres() to add a Postgres database to your app and deploy it with azd, azd uses the Postgres dev service feature in ACA to give you a database. That's great for development because it spins up fast and is cheap to run. But in production you might rather use something like Azure Database for PostgreSQL - Flexible Server to offload the operations of the server to managed service so you don't have to play SRE (and you get better performance and durability). Currently we are thinking that might look something like this builder.AddPostgres().WithMetadata("azd-postgres-server-type", "flexible-server"). From this loosely typed interface primitive we could expect building a strongly typed interface over this. This would still not expose every knob of every service - and so we are also thinking about things like "can you give us a diff to apply to the object that we'd send to the ARM control plane to create the resource. It's less clear to me right now how that would work - but we do know this is an issue and are trying to figure out a solution. We are also thinking about how to improve infra synth to allow to you just create IaC for parts of your application (e.g. for just a few resources) so you can hand edit these parts and let azd + Aspire auto-generate and handle the rest of stuff.

We also have been talking about (3) and it's something we want to deliver as part of Aspire GA. We have some ideas here and you should see that work come online during the preview 4 timeframe as well.

All of this work can be done without adopting Pulumi and we plan to build all this stuff on top of Bicep.

A place where Pulumi would be helpful is if you wanted to start to manage resources in non Azure clouds (since azd will always be focused on Azure) or you cared about using Pulumi instead of the ARM Deployment (perhaps you want to be able to use esc to manage configuration, perhaps you want to buy into their Policy as Code story, or use their RBAC support for organizations to control who can deploy to what stacks). For us to make this work we'd have to two major things:

Add support to azd provison to use Pulumi instead of ARM to do a deployment. This would be similar to our (beta) support for using terraform with azd.
Allow azd to generate a Pulumi program to describe your Aspire application and hand that to pulumi during azd provision. We could either do this by modifing azd to also be able to generate Pulumi code in addition to Bicep or do something like generate the bicep and then run a converter to turn it into Pulumi.

This is a large amount of work (and adding another IaC provider to azd has some higher than expected costs, since IaC provider ends up having impacts on the rest of the system and represents a non-trivial cost to maintain over time) and so as a small team right now we are focused on really trying to nail the "Azure Native" version built on top of Bicep. We also selfishly want to use it as a way to highlight places where we can improve the developer experience of folks who work with Azure day to day so we can drive improvements to the platform.

If I was to put my Pulumi hat on I think the answer would be something like: "There should be a Pulumi component that let's you do something like: new AspireApp("<path-to-app-host-project>"); and the implementation of that component talks to the app host to extract its manifest and then "compiles" that to a set of Pulumi resources." If I was still on the Pulumi team, that is probably what I would be looking at building.

But that's a discussion for another day and another issue (I will try to find some time to brain dump thoughts into that discussion thread at some point).

Hope this is helpful and gives you an idea about what we are thinking about and "where the puck is headed". Sorry for taking a while to get back to you - you're touching on the some of the core complex issues we've been struggling with and so it took some time for me to synthesize our current thinking and lay it all out for you (while at the same time trying to make azd better and work through the design meetings around these issues).

Hope this is helpful - happy to answer any follow ups, but it might take me some time :-).

VincentH-Net commented 7 months ago

Thank you @ellismg - I really appreciate the effort to share this comprehensive overview. It took some time for me as well to let it sink in and reciprocate. The examples you mention match my experience; sharing this is very helpful in wrapping one's mind around the technologies, and it can help to determine a course in solution technology roadmaps.

my general view is that Aspire is/can be this (= a full fledged IaC solution)

Hmmm... my grok (so far) is that Aspire helps simplify multiple aspects of distributed application development (IaC being one), while Pulumi is fully focused on IaC. Think e.g. about all the different environments a solution exists in with their respective infrastructure variations and partial IaC reuse across them, versus just two infrastructure variations: local and deployed.

As a .NET / Azure distributed application architect I like to have as much power as possible, as soon as possible, with early maturity, to reduce complexity and effort. A clear product focus helps for that; from that perspective it would appeal to me to have Aspire become truly great for development-centric aspects of distributed applications, including the first step / core IaC, and focus on a smooth and flexible integration with specialized infrastructure solutions such as Pulumi, rather than trying to make Aspire a full IaC competitor. As seen on many occasions, combining strengths and clear focus makes an ecosystem more valuable than adding more alternatives that have a bigger overlap with a less intense focus. It results in earlier maturity as well.

Just food for thought... not disagreeing with you here.

That being said, the Azure focus of AZD is clear and I think some of the features described above would be great:

builder.AddAzureDeployment("mycustomresource", <path-to-a-bicep-or-arm-template>) seems like an excellent first step. Generating full Azure coverage for Aspire as well. Preview 4 (of Aspire) is quite close too!
builder.AddPostgres().WithMetadata("azd-postgres-server-type", "flexible-server") this one raises some concerns - it seems hard to discover and debug, hard to support in IDE, and has the potential to create the type of "magic configuration strings" pain that plagues today's distributed application development tooling, and that Aspire set out to solve in the first place. This also touches upon my "Hmmm" above - perhaps this would be better served with providing a frictionless handover to specialized IaC? That could be something the Aspire/AZD teams build, but it could also be an active collaboration with Pulumi et al to stimulate and help them use Aspire as input.
I wonder which would be a better integration point - deployed infrastructure or the template / manifest / program that created it? Is the goal to support manually deployed / (partly) configured resources?

In short, I love the plans in 1 to address gaps 1 and 2 - I will be using that functionality for sure. Plan 2 I would not do and instead try to smooth integration (incoming and/or outgoing) with other IaC products. That might also obviate the need for plan 3.

Hope this helps - interested to continue the evolution of thinking, and thanks again for sharing!

VincentH-Net commented 7 months ago

My point at

Hmmm...

above prompted me to start a wider discussion about IaC in the Aspire repo; afaiac we can continue with other AZD specific things below; hth

cveld commented 7 months ago

Quick reference to the Radius project: https://github.com/radius-project/radius Radius ultimately provides a separation of concerns between your infrastructure definition and the app workload.

Azure / azure-dev

Add support for Pulumi #93