kestra-io / kestra

:zap: Workflow Automation Platform. Orchestrate & Schedule code in any language, run anywhere, 500+ plugins. Alternative to Zapier, Rundeck, Camunda, Airflow...
https://kestra.io
Apache License 2.0
11.06k stars 952 forks source link

[Outside of our control] Change the directory to the current WORKING_DIR for AWS Batch and Google Batch task runners #3493

Open anna-geller opened 6 months ago

anna-geller commented 6 months ago

Feature description

Azure already works

TODO: cd to the workingDir after starting the taskrun container/process

EDIT: likely will never be done:

  1. Reference issue at GCP side: https://github.com/googleapis/google-cloud-java/issues/10670
  2. No mechanism to track Batch feature requests open to the public other than keeping an eye on the changelog
loicmathieu commented 6 months ago

After more thoughts, it can cause issues for images where the entry point is not a shell (for ex AWS CLI).

So we decided to warn in the description for now, we open issues on both AWS and GCP side to be able to fix it upstream.

The following sentence has been added to both description:

Warning, contrarily to other task runners, this task runner didn't run the task in the working directory but in the root directory. You must use the {{workingDir}} Pebble expression or the WORKING_DIR environment variable to access files.

loicmathieu commented 6 months ago

Reference issue at AWS side: https://github.com/aws/aws-sdk/issues/716

loicmathieu commented 6 months ago

Reference issue at GCP side: https://github.com/googleapis/google-cloud-java/issues/10670