search

kestra-io / plugin-scripts

https://kestra.io/plugins/
Apache License 2.0
9 stars 11 forks source link

io.kestra.plugin.scripts.python.Commands - containerImage dynamic feature not working #147

Closed japerry911 closed 6 months ago

japerry911 commented 6 months ago

Expected Behavior

...
  - id: fetch_data
    type: io.kestra.plugin.scripts.python.Commands
    beforeCommands:
      - export PYTHONPATH=src
      - export GCP_PROJECT_ID={{ secret('GCP_PROJECT_ID') }}
    commands:
      - python {{ inputs.fetch_command }} --uuid_str {{ inputs.uuid_str }}
    containerImage: "us-central1-docker.pkg.dev/{{ secret('GCP_PROJECT_ID') }}/kestra/kestra-woohoo-v2:latest"
...

The image utilizes the following container static container image in Batch Job: us-central1-docker.pkg.dev/{{ secret('GCP_PROJECT_ID') }}/kestra/kestra-woohoo-v2:latest , no secret substitution.```yaml ...

expected container image:

us-central1-docker.pkg.dev/my-gcp-project-id/kestra/kestra-woohoo-v2:latest

Actual Behaviour

The image utilizes the following container static container image in Batch Job: us-central1-docker.pkg.dev/{{ secret('GCP_PROJECT_ID') }}/kestra/kestra-woohoo-v2:latest , no secret substitution.

Steps To Reproduce

  - id: fetch_data
    type: io.kestra.plugin.scripts.python.Commands
    beforeCommands:
      - export PYTHONPATH=src
      - export GCP_PROJECT_ID={{ secret('GCP_PROJECT_ID') }}
    commands:
      - python {{ inputs.fetch_command }} --uuid_str {{ inputs.uuid_str }}
    containerImage: "us-central1-docker.pkg.dev/{{ secret('GCP_PROJECT_ID') }}/kestra/kestra-woohoo-v2:latest"
    warningOnStdErr: false
    taskRunner:
      type: io.kestra.plugin.gcp.runner.GcpBatchTaskRunner  
 ....

Environment Information

Example flow

  - id: fetch_data
    type: io.kestra.plugin.scripts.python.Commands
    beforeCommands:
      - export PYTHONPATH=src
      - export GCP_PROJECT_ID={{ secret('GCP_PROJECT_ID') }}
    commands:
      - python {{ inputs.fetch_command }} --uuid_str {{ inputs.uuid_str }}
    containerImage: "us-central1-docker.pkg.dev/{{ secret('GCP_PROJECT_ID') }}/kestra/kestra-woohoo-v2:latest"
    warningOnStdErr: false
    taskRunner:
      type: io.kestra.plugin.gcp.runner.GcpBatchTaskRunner  
 ....
tchiotludo commented 6 months ago

@loicmathieu : you could make a full review of taskrunner please with dynamic vars, seems that there is a lots missing, thx