Overriding the `kestra.tasks.defaults` config property doesn't work

[yuri1969]

Explain the bug

As the docs state, a non-forced task defaults should be applied only when the task itself does not define such property.

This doesn't seem to apply on the io.kestra.plugin.scripts.shell.Commands's runner property.

---

Snippet of config file which defaults the runner type to PROCESS for the each shell.Commands task:

kestra:
  tasks:
    defaults:
      - type: "io.kestra.plugin.scripts.shell.Commands"
        values:
          runner: PROCESS

Flow overriding the cluster default runner type to DOCKER:

id: hello-world-from-docker
namespace: company.team

tasks:
  - id: use-docker
    type: io.kestra.plugin.scripts.shell.Commands
    runner: DOCKER # Use the Docker runner
    commands:
      - echo "Hello from Docker!"

---

Executing such flow still results in a local call - not using Docker.

It seems this issue might be related to #2364.

Environment Information

• Kestra Version: 0.14.0-SNAPSHOT as of Jan 3rd 2024
• Operating System and Java Version (if not using Kestra Docker image): Corretto17

[brian-mulier-p]

I guess it's the same issue as https://github.com/kestra-io/kestra/issues/2364 See https://kestra-io.slack.com/archives/C04HTFM3VL6/p1709724906043089 but there is 2 issues to tackle:

• https://github.com/kestra-io/kestra/blob/c15e971ed6884cfb8b4f836aac43ea4a75a39fd2/cli/src/main/resources/application.yml#L70 which leads to some property erasure in case of default values are provided (which means we have no clue to know if a default value was explicitly supplied by user)
• https://github.com/kestra-io/kestra/blob/c15e971ed6884cfb8b4f836aac43ea4a75a39fd2/core/src/main/java/io/kestra/core/services/TaskDefaultService.java#L87 we end up in the TaskDefaultService with a Flow already parsed, which means that initial assignments (member declaration with assignment) already took place in case of empty value (which is the case due to the above non-default serialization) and leads to unability to know if a default value was explicitly supplied

TLDR: It's a hard subject with some major reworks which won't be tackled this release

[loicmathieu]

The issue is indeed tricky: task properties having a default value (like runner or pullPolicy using @Builder.Default in the Java code) cannot be set in a flow if there is a taskDefaults defined in the Kestra configuration file with a different value that the initial property.

It's also the case with taskDefaults defined at the flow level, for ex:

id: hello-world-from-docker
namespace: company.team

taskDefaults:
  - type: io.kestra.plugin.scripts.shell.Commands
    values:
      runner: PROCESS

tasks:
  - id: use-docker
    type: io.kestra.plugin.scripts.shell.Commands
    runner: DOCKER # Use the Docker runner
    commands:
      - echo "Hello from Docker!"

We will soon switch to serialization including NON_NULL but it's only the beginning of the story, it will not fully fix the issue.

[loicmathieu]

Same root cause as https://github.com/kestra-io/kestra/issues/2260