Parameterised Tasks

[nicornk]

Problem description

I would like to include deployment related tasks into my pixi task configurations. Usually, these tasks have parameters with default values that are overwritten in a CI/CD Pipeline or when executing locally.

I do not find a good way (without custom shell scripts) to define environment variables with default values since the Deno Task Shell does not support Parameter Expansion https://www.gnu.org/software/bash/manual/bash.html#Shell-Parameter-Expansion

✨ Pixi task (default): export VAR=asdf && echo ${VAR-defaultValue}
  × deno task shell failed to parse 'export VAR=asdf && echo ${VAR-defaultValue} ': Expected command following boolean operator.
  │ 
  │ Unexpected character.
  │   {VAR-defaultValue}
  │   ~

One possibility would be if parameters are a new key in the tasks and are setup as env. variables by pixi:

deploy = { cmd = [
    "aws",
    "cloudformation",
    "deploy",
    "--template-file",
          "cfn/cfn-template.yml", 
    "--stack-name",
          "$STACK_NAME_PARAM",
    "--parameter-overrides",
          "SomeParameterOverride=$OTHER_PARAM",

] , params = [{key: "STACK_NAME_PARAM", default: "batch-job", key: "OTHER_PARAM", default: "foo" }]
}

which would resolve to aws cloudformation deploy --template-file cfn/cfn-template.yml --stack-name batch-job --parameter-overrides SomeParameterOverride=foo when called with pixi run deploy

or to aws cloudformation deploy --template-file cfn/cfn-template.yml --stack-name overwrite --parameter-overrides SomeParameterOverride=foo when called with STACK_NAME_PARAM=overwrite pixi run deploy

Thanks!

[ruben-arts]

Related issue but not the same: #957

[mjkanji]

I have a related problem with wanting to check that a given environment variable is set before running my command. For example,

[tasks]
my-task = { cmd = [
    "some-command",
    "--some-option",
    "\"models/$PROJECT/file.py\"",

] }

I can parametrize the value of PROJECT using env vars as follows:

PROJECT=my_project pixi run my-task

However, this requires PROJECT to be set beforehand, otherwise I get an invalid path: models//file.py. Is there any way to force the deno_task_shell to error out if a given env var is not set? I tried adding "set -u to the command, but that didn't really work.

[ruben-arts]

There currently isn't a way to test the environment variables in deno_task_shell thus pixi. This is good input for the design of parameterized tasks.

[nicornk]

I think https://github.com/prefix-dev/pixi/pull/972 solves this issue? Will give it a try

[W1M0R]

The following method may also work: https://taskfile.dev/usage/#forwarding-cli-arguments-to-commands

If -- is given in the CLI, all following parameters are added to a special .CLI_ARGS variable. This is useful to forward arguments to another command.

In pixi, this would translate to something like:

[tasks]
my-task = { cmd = [
    "some-command",
    "--some-option",
    "$CLI_ARGS",
] }

pixi run my-task -- models/my_project/file.py

If you want to call my-task (with some known hard-coded arguments) from another task, you could then explicitly set the CLI_ARGS variable for the task when calling it, like this: https://taskfile.dev/usage/#calling-another-task. This would work if the depends-on field accepts an env property for items.

Some related issues:

1. https://github.com/prefix-dev/pixi/issues/502