search

pulumi / pulumi-yaml

YAML language provider for Pulumi
Apache License 2.0
38 stars 11 forks source link

feat: add PULUMI_STACK for compiler run enviroment #578

Closed phramz closed 1 month ago

phramz commented 1 month ago

This PR adds a new environment variable PULUMI_STACK to the compiler run.

The possibility to determine the stack from the compiler makes it a lot more flexibel when dealing with stack specific resources.

Here an example (I'm using Jinja2 though the issue itself is not compiler specific):

# Pulumi.yaml
runtime:
  name: yaml
  options:
    compiler: jinja2 --format=yaml Pulumi.yaml values.yaml

 resources:
  # stack specific resources for stage
  # {% if 'stage' == 'PULUMI_STACK'|getenv %}{% include 'templates/mailcatcher.yaml.j2' %}{% endif %}

  # common resources

A more complex scenario would be to add HA resources to a production stack where an integration system is zonal set up to reduce cloud costs.

My approach seems a bit naive to be honest, but I couldn't figure out a better way without breaking interfaces. Also this solution only works from the Run-call context, so GetRequiredPlugins might not be accurate when plugins are stack specific.

WDYT?

github-actions[bot] commented 1 month ago

PR is now waiting for a maintainer to take action.

Note for the maintainer: Commands available:

github-actions[bot] commented 1 month ago

PR is now waiting for a maintainer to take action.

Note for the maintainer: Commands available:

phramz commented 1 month ago

Ideally we would also have a test showing that this works.

Regarding a test ... I'm not sure where and how to ideally implement such a test. Would be an acceptance test unless we can mock the exec package somehow (probably a little refactoring).

Any thoughts or ideas on this @tgummerer

tgummerer commented 1 month ago

I think an integration test would be fine here, maybe creating two different stacks with different outputs based on the environment variables you are introducing here? pkg/tests/integration_test.go has some examples of current integration tests.

github-actions[bot] commented 1 month ago

PR is now waiting for a maintainer to take action.

Note for the maintainer: Commands available:

phramz commented 1 month ago

I think an integration test would be fine here, maybe creating two different stacks with different outputs based on the environment variables you are introducing here? pkg/tests/integration_test.go has some examples of current integration tests.

@tgummerer I'm having a hard time setting up the integration tests to run locally. Obviously Pulumi is not using my updated binary as language plugin. So my changes don't have any effect during the test. Is there a doc or could you give me a hint on how to run them locally.

I already pushed an unfinished test PTAL.

Installing the binary will give me an error:

error: the yaml language plugin is bundled with Pulumi, and cannot be directly installed with this command. If you need to reinstall this plugin, reinstall Pulumi via your package manager or install script.
tgummerer commented 1 month ago

/run-acceptance-tests

github-actions[bot] commented 1 month ago

Please view the results of the PR Build + Acceptance Tests Run Here

tgummerer commented 1 month ago

Installing the binary will give me an error:

How exactly were you trying to install the binary? I believe you should be able to run make install to install the binary (and then prepend $GOBIN to the $PATH.) There's also a make test Makefile target that should run all tests. Does that not work for you?

github-actions[bot] commented 1 month ago

PR is now waiting for a maintainer to take action.

Note for the maintainer: Commands available:

phramz commented 1 month ago

Installing the binary will give me an error:

How exactly were you trying to install the binary? I believe you should be able to run make install to install the binary (and then prepend $GOBIN to the $PATH.) There's also a make test Makefile target that should run all tests. Does that not work for you?

Thanks! It was indeed a PATH issue having both Pulumi from source and Brew in place.

I finished/fixed the test, fingers crossed it'll pass now.

github-actions[bot] commented 1 month ago

PR is now waiting for a maintainer to take action.

Note for the maintainer: Commands available:

Frassle commented 1 month ago

/run-acceptance-tests

github-actions[bot] commented 1 month ago

Please view the results of the PR Build + Acceptance Tests Run Here

github-actions[bot] commented 1 month ago

PR is now waiting for a maintainer to take action.

Note for the maintainer: Commands available:

tgummerer commented 1 month ago

/run-acceptance-tests

github-actions[bot] commented 1 month ago

Please view the results of the PR Build + Acceptance Tests Run Here

github-actions[bot] commented 1 month ago

PR is now waiting for a maintainer to take action.

Note for the maintainer: Commands available:

tgummerer commented 1 month ago

/run-acceptance-tests

github-actions[bot] commented 1 month ago

Please view the results of the PR Build + Acceptance Tests Run Here

github-actions[bot] commented 1 month ago

PR is now waiting for a maintainer to take action.

Note for the maintainer: Commands available:

github-actions[bot] commented 1 month ago

PR is now waiting for a maintainer to take action.

Note for the maintainer: Commands available:

tgummerer commented 1 month ago

/run-acceptance-tests

github-actions[bot] commented 1 month ago

Please view the results of the PR Build + Acceptance Tests Run Here

phramz commented 1 month ago

Thank you again, and thanks for the patience while we worked out the best way to also make this backwards compatible.

I'm gonna leave this PR open until tomorrow just in case anyone else has any comments, but this looks good to me!

That went really smooth, kudos and big thanks to you!