spring-cloud / spring-cloud-dataflow

A microservices-based Streaming and Batch data processing in Cloud Foundry and Kubernetes
https://dataflow.spring.io
Apache License 2.0
1.1k stars 578 forks source link

TASK_EXECUTION_ID not propagated by SCDF in entryPointStyle boot #5917

Open jnt0r opened 2 weeks ago

jnt0r commented 2 weeks ago

Description: When changing the entryPointStyle to boot the Variable SPRING_CLOUD_TASK_EXECUTIONID is not present in the SPRING_APPLICATION_JSON environment variable. When using the shell entryPointStyle, the ENV SPRING_CLOUD_TASK_EXECUTIONID is present with the correct value. This results in the launched task to generate new TaskExecutionId and therefore the id in SCDF and the TASK do not match. Also there are always 2 task executions when launching a single task.

Release versions: { "versions": { "implementation": { "name": "spring-cloud-dataflow-server", "version": "2.11.3" }, "core": { "name": "Spring Cloud Data Flow Core", "version": "2.11.3" }, "dashboard": { "name": "Spring Cloud Dataflow UI", "version": "3.4.3" }, "shell": { "name": "Spring Cloud Data Flow Shell", "version": "2.11.3", "url": "https://repo.maven.apache.org/maven2/org/springframework/cloud/spring-cloud-dataflow-shell/2.11.3/spring-cloud-dataflow-shell-2.11.3.jar" } }, "features": { "streams": true, "tasks": true, "schedules": true, "monitoringDashboardType": "NONE" }, "runtimeEnvironment": { "appDeployer": { "deployerImplementationVersion": "2.11.3", "deployerName": "Spring Cloud Skipper Server", "deployerSpiVersion": "2.11.3", "javaVersion": "21.0.3", "platformApiVersion": "", "platformClientVersion": "", "platformHostVersion": "", "platformSpecificInfo": { "default": "kubernetes" }, "platformType": "Skipper Managed", "springBootVersion": "2.7.18", "springVersion": "5.3.34" }, "taskLaunchers": [ { "deployerImplementationVersion": "unknown", "deployerName": "KubernetesTaskLauncher", "deployerSpiVersion": "unknown", "javaVersion": "21.0.3", "platformApiVersion": "v1", "platformClientVersion": "unknown", "platformHostVersion": "unknown", "platformSpecificInfo": { "namespace": "XXX", "master-url": "XXX" }, "platformType": "Kubernetes", "springBootVersion": "2.7.18", "springVersion": "5.3.34" } ] }, "monitoringDashboardInfo": { "url": "", "source": "default-scdf-source", "refreshInterval": 15 }, "security": { "isAuthentication": false, "isAuthenticated": false, "username": null, "roles": [] }, "git": { "commit": "2183849" } }

Steps to reproduce: switch to boot entryPointStyle and start a task.

The resulting value of SPRING_APPLICATION_JSON: { "management.metrics.tags.service": "task-application", "spring.datasource.url": "XXX", "spring.datasource.driverClassName": "XXX", "management.metrics.tags.application": "${spring.cloud.task.name:unknown}-${spring.cloud.task.executionid:unknown}", "spring.cloud.task.initialize-enabled": "false", "spring.batch.jdbc.table-prefix": "BOOT3_BATCH_", "spring.cloud.task.schemaTarget": "boot3", "spring.cloud.task.name": "XXX", "spring.cloud.task.tablePrefix": "BOOT3_TASK_", "spring.cloud.deployer.bootVersion": "3" }

corneil commented 2 weeks ago

From my tests is seems that when using boot or exec the property spring.cloud.task.executionid added as argument regardless of the entryPointStyle. Only with shell is it added as an environmental variable.

It seems that in the case of boot the SPRING_APPLICATION_JSON has already been created when we obtain new id and add argument.