Support of the `environment` resource?

[mathieu-benoit]

If doing this:

apiVersion: score.dev/v1b1
metadata:
  name: my-sample-workload
containers:
  my-sample-container:
    image: .
    variables:
      ENV_VAR: ${resources.env.my-env-var}
resources:
 env:
  type: environment

Getting this error:

Error: failed to provision resources: resource 'environment.default#my-sample-workload.env' is not supported by any provisioner

While it's supported by score-compose by default: https://github.com/score-spec/score-compose/blob/main/internal/provisioners/envprov/envprov.go.

[sujaya-sys]

@mathieu-benoit at the community meeting last week we discussed the following: Until we have a clear idea of how to interpret "environment" in score-k8s (possibly a configmap?) we'd recommend users to implement custom resource provisioners based on their use case. The suggestion was to update the error message and make this more clear.

For example

Error: failed to provision resources: The resource 'environment.default#my-sample-workload.env' is not supported by any existing provisioner. Please implement a custom resource provisioner to support this resource type.

Curious to hear what you think!

[maxstepanov]

@sujaya-sys I've tried implementing this with a custom provisioner and failed. I bootstrapped state.yaml with a shared variable environment

shared_state:
  environment: develop

then tried loading a file with the same name

- uri: cmd://bash
  type: environment
  args:
  - -c
  - |
    echo '{"resource_outputs":{'
    while IFS=': ' read -r key value; do
      echo "\"$key\": \"$value\""
    done < {{ .Shared.environment }}.env
    echo '},"manifests":[]}'

bash: -c: line 4: syntax error near unexpected token `.Shared.environment'

Sadly doesn't work, surprisingly because there is no templating in cmd:// default provisioner. Actions in template provisioners don't seem to support loading from files anything other than other templates

How would you recommend to implement this?

[sujaya-sys]

@astromechza any chance you could have a quick look at this and provide some guidance here? Thanks so much!

[astromechza]

@maxstepanov to setup a custom environment provisioner, you can add a .score-k8s/custom.provisioners.yaml with your custom provisioner, and then declare it in your score file as

resources:
  env:
    type: environment

You should not need to modify state.yaml 🤔

What was the goal with the done < {{ .Shared.environment }}.env in your example above?

I don't think we should support templating in the cmd provisioners because it's a security concern. It allows values in the Score workload to change what binaries and arguments are executed rather than the ones intended by the team providing the custom provisioners file.

[maxstepanov]

@astromechza i was trying to implement loading of environment variables from files, .env style and wasn't aware that CMD args doesn't support templating. score-k8s doesn't implement environment provisioner and doesn't support passing --env-file. So i didn't find a better way than to stick environment name as .Shared in state.yaml manually. Then my environment provisioner can make the right decision which file to load. Is there any better way to accomplish something like that?

Should score-k8s have an environment provider and support --env-file cli argument or is it left to the user to implement using cmd provider?

In case of a CMD provider route how would it figure out which file to read? Is there a way to pass it via cli? With the lack of metadata in state.yaml, .Shared the only place i found to put it in.

[astromechza]

The guidance is that you create an environment-specific provisioners file, like:

develop.provisioners.yaml

- uri: template://develop-env
  type: environment
  outputs:
    KEY: VALUE
    KEY2: VALUE2

Or if you want to cat a file on disk

- uri: cmd://bash
  type: environment
  args:
  - -c
  - "cat ./develop.env"

where develop.env contains

{"resource_outputs":{
  "KEY": "VALUE",
  "KEY2": "VALUE2"
}}

Then you use your target environment to add a different ENV.provisioners.yaml file into the .score-k8s/ directory.

[maxstepanov]

Thank you for the guidance. First option is no go for us. Not gonna force developers to write provisioners. The second option is feasible. This issue seems to be a documentation issue since there is no intent to support environment resource out of the box.

On Wed, Sep 4, 2024, 20:14 Ben Meier @.***> wrote:

The guidance is that you create an environment-specific provisioners file, like:

develop.provisioners.yaml

• uri: template://develop-env type: environment outputs: KEY: VALUE KEY2: VALUE2

Or if you want to cat a file on disk

• uri: cmd://bash type: environment args:
  • -c
  • "cat ./develop.env"

where develop.env contains

{"resource_outputs":{ "KEY": "VALUE", "KEY2": "VALUE2" }}

Then you use your target environment to add a different ENV.provisioners.yaml file into the .score-k8s/ directory.

— Reply to this email directly, view it on GitHub https://github.com/score-spec/score-k8s/issues/24#issuecomment-2329592348, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAESDHZ6YWZVC7ZDEOOWWNLZU45VTAVCNFSM6AAAAABLQ2JAH2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDGMRZGU4TEMZUHA . You are receiving this because you were mentioned.Message ID: @.***>

[astromechza]

@maxstepanov unfortunately there is no default behavior here that works for everyone, so a custom environment provisioner is necessary. It only needs to be written once, and installed as part of your score init process.

Here's a working example that loads a .env file:

- uri: cmd://python3#dotenv
  type: environment
  args:
  - -c
  - |
    import json
    with open('.env') as f:
      content = f.read().strip()
    env_map = dict([x.split("=",1) for x in content.strip().splitlines()])
    print(json.dumps({"resource_outputs": env_map}))

So I run score-k8s init and copy the content into a custom-provisioner file. (I'm working on a quick install flag here now).

[maxstepanov]

@astromechza how to handle the following situation. In our case resource names are prefixed with environment name. Our developer reference the canonical name of the resource and rendering figures out what is the actual name in the context of the environment.

I'm using .Shared.environment value which i put manually in. It's either this or template the template provisioners(having a separate provisioner per environment). .score-k8s directory seems to be what defines the environment. May be it's possible to add some sort of metadata in score.yaml which is accessible to provisioners?

Here is my pubsubtopic resource template.

- uri: template://custom/pubsubtopic
  type: pubsubtopic
  init: |
    {{ if not .Params.name }}{{ fail "params.name must be specified" }}{{ end }}
  state: |
    {{ if not .State.name }}
    name: "{{ .Shared.environment }}-{{ .Params.name }}-{{ randAlpha 6 | lower }}"
    {{ else }}
    name: "{{ .State.name }}"
    {{ end }}
  outputs: |
    name: "{{ .State.name }}"
  manifests: |
    - apiVersion: pubsub.cnrm.cloud.google.com/v1beta1
      kind: PubSubTopic
      metadata:
        name: "{{ .State.name }}"

Is there a better way?

[astromechza]

@maxstepanov generally your provisioners define the environment, so you'd pull in different provisioners based on your environment. In Humanitec for example, they can use "matching criteria" to link different provisioners based on the environment. Humanitec also supports hierarchical provisioners, so resources can refer to the outputs of other resources. So far we don't support this in any of the Score implementations just yet.

[maxstepanov]

@astromechza thank you. it's much clearer now.

Humanitec also supports hierarchical provisioners, so resources can refer to the outputs of other resources.

That was kinda my next question. Support for shared resources between workloads. Basically using outputs of resources from a different workload in the same state. I guess this is for separate issue.