"No code" New Project experience

[lukehoban]

We've talked to many users who would like a templates/service-catalog experience that doesn't require end users to manage code. Specifically:

1. They would like to be able to create and deploy Pulumi templates with just clicks in the Pulumi Console
2. They would like to not require users to put the code in a new repo or think about code at all.

Today, the New Project Wizard requires that an instantiated template's code be put somewhere and managed by the end user of the template. There are two options today:

1. Run pulumi new and get a copy of the code locally - user has to decide where to commit and version that code (either just on local machine, or commiting/pushing into a new or shared repo and connecting to CI long term).
2. Create a new repository with the code, and configure Pulumi Deployments to deploy from that repository on push.

To achieve the goals here - we want something similar to (2), but without creating a new repo.

There are a number of questions:

1. What does Deployments point at if not a new repo? Does it point at the template repo directly? Does it point at some storage for the project/stack that lives inside the Pulumi Cloud?
2. Templates support templating (modifications to the code as part of instantiated it). How is that supported if there is not separate copy of the instantiated template?
3. Pulumi new requires writing a config file as part of instantiating the template. Where does that config get stored? Do we need service-managed config to support this?
4. How does the user update the stack? If they want to pick up a new version of the template for example? This won't be triggered by a Git workflow, so what workflow is it driven by? Presumably new UI and API in the stack page for stacks that are setup in this way?
5. Related to the above - does this need to be a "different" kind of stack, such that we don't try to expose all the Git/code details of the stack? Or is it just a normal stack whose updates are triggered in a different say?
6. We do have an existing pulumi up <url> support in the CLI, which has a lot of similarities to this (the user of pulumi up <url> doesn't have to have any code - just a URL they want to deploy) - can we build on top of that somehow?

[EvanBoyle]

What does Deployments point at if not a new repo?

I'll throw out some other options here.

1. a component + version (Though there is more friction if you either have to publish to something like NPM, or publish an entire MLC to get started)
2. a pre-existing repo with source code and a Pulumi.yaml file (no need to push a pulumi.stack.yaml file into the repo, just point at it for source code)
3. teach deployments to understand a template as a new source context and use pulumi new -g within the "acquire source" step. This is sort of similar to the pulumi up --url that you suggest later on.

Where does that config get stored?

Service-managed config (ie being able to edit the config on the stacks page) would probably be the most intuitive. Though technically, a stack-level ESC environment could store all config for the stack and offer a way to edit it after the fact as well. The only bit that we'd need to ensure is that pulumi config refresh would also refresh the environment: section of the stack file as well.

How does the user update the stack?

A broader question here is versioning. Git tags on a template or source repo might be an intuitive way to achieve this and users have asked for git tag support already https://github.com/pulumi/pulumi-cloud-requests/issues/328. I do think we also need to think through the workflow of updating config and subsequently updating the stack. Will users click to deploy updates after editing config or changing the version of a template/component/whatever we depend on for this scenario?

[joeduffy]

One important point just to make clear, the target audience for this is not the end users themselves, even though they will use the product, it is platform teams who have to cater to a varying degree of end user sophistication when it comes to IaC. One very common pattern is “end user should not interact with IaC directly”; eg just let them spin up a Kubernetes cluster. Our customer here is the platform team who otherwise needs to build a ton of scaffolding to make this work securely and reliably in a custom way and frankly most of the time simply won’t since they get this easier platform experience out of the box with many other vendors.

[joeduffy]

teach deployments to understand a template as a new source context and use pulumi new -g within the "acquire source" step. This is sort of similar to the pulumi up --url that you suggest later on.

Git tags on a template or source repo might be an intuitive way to achieve this and users have asked for git tag support already https://github.com/pulumi/pulumi-cloud-requests/issues/328

I imagined something like this. I don't think we need to store any code anywhere beyond just the template repo and an optional committish if people want to pin it to a version. It seems pretty straightforward to just have deploys of these things clone the template, twiddle whatever's necessary to make it runnable (if anything), and plop a config YAML in the workdir.

a stack-level ESC environment could store all config for the stack

I love having integrations with ESC, but I imagined that to be an optional integration, just as with templates today. In other words, I'd hate to strangely start layering in opinions, versus continuing to have orthogonal configurable axes.

[EvanBoyle]

and plop a config YAML in the workdir

love having integrations with ESC, but I imagined that to be an optional integration

Is the config editable after the user goes through the wizard for the first time? If not, no problem, but if so we need to store it somewhere and think about the UX for updating it. We already have this with ESC which is why I bring it up, though I understand your concerns. Some other options for storing config that I can think of:

• prerun commands (clunky, poor editing experience for the less IaC savvy end user, no support for secret prerun commands)
• environment variables (template config and the related forms in the NPW can't currently write to env vars, it is all build around config)
• build service managed config

[joeduffy]

Yes it would be editable. I assumed we would use something like Pulumi config refresh. That said in many ways I care way more about the user experience than how it’s built. If we needed a new table of config key/values, that doesn’t seem incredibly onerous to me.

[joeduffy]

That said I assume what’s actually editable is not the config per se, but the template parameters.

[cleverguy25]

Per my conversations with Mitch and also from working on platform teams and the gitops trends, is repoless really what they want or do they want to repo to fade into the background? For example we could have config that configures a central repo and folder structure and have the NPW push to that repo without the user having to think about repos at all but the enterprise would still have direct access to the code. I would think in the end this is what enterprise customers really want.

That said allowing UI to manage stack config for teams that do trust Pulumi to be the source of truth for both code and config is not unreasonable and is a valuable scenario.

I would hate to see us put too much energy into full repoless to find out they really want a repo at the end.