Closed younsl closed 11 months ago
cppwfs
commented
1 year ago CTR is still a boot2 application. The apps that it launches can be boot 3 or boot 2.
If I understand your settings, launch CTR without the following properties:
--spring.cloud.task.schemaTarget=boot3
--spring.cloud.task.tablePrefix
--spring.cloud.deployer.bootVersion=3
younsl
commented
1 year ago It's not worked for me.
My environment is SCDF v2.11.1 on EKS v1.24.
I deleted 3 application properties when I execute composed task:
spring.cloud.deployer.bootVersion = 3spring.cloud.task.tablePrefix = BOOT3_TASK_spring.cloud.task.schemaTarget = boot3After that:
I also deleted related settings from the ConfigMap used by the SCDF server pod.
spring.cloud.deployer.bootVersion = 3spring.cloud.task.tablePrefix = BOOT3_TASK_spring.cloud.task.schemaTarget = boot3Data
====
application.yaml:
----
spring:
security:
oauth2:
client:
registration:
google:
client-id: <REDACTED>-<REDACTED>.apps.googleusercontent.com
client-secret: <REDACTED>
scope:
- profile
- email
redirect-uri: https://<REDACTED>/login/oauth2/code/google
authorization-grant-type: authorization_code
provider:
google:
user-info-uri: https://www.googleapis.com/oauth2/v3/userinfo
user-name-attribute: email
cloud:
dataflow:
security:
authorization:
provider-role-mappings:
google:
map-oauth-scopes: false
task:
platform:
kubernetes:
accounts:
default:
environmentVariables:
- <REDACTED>=dev
limits:
cpu: 500m
memory: 1024Mi
readinessProbeDelay: 120
livenessProbeDelay: 90
podSecurityContext:
runAsUser: 1001
entryPointStyle: exec
task:
closecontextEnabled: true
datasource:
url: '...'
driverClassName: org.mariadb.jdbc.Driver
username: DataFlowServer
password: ${DATABASE_PASSWORD}
testOnBorrow: true
validationQuery: "SELECT 1"
flyway:
enabled: trueAnd I clicked [launch task] button on SCDF web UI.
Deleted related properties and launch composed-task, then CTR failed too.
Fail logs of composed-task pod:
scdf-composed-task-runner 14:09:55.85
scdf-composed-task-runner 14:09:55.85 Welcome to the Bitnami spring-cloud-dataflow-composed-task-runner container
scdf-composed-task-runner 14:09:55.85 Subscribe to project updates by watching https://github.com/bitnami/containers
scdf-composed-task-runner 14:09:55.85 Submit issues and feature requests at https://github.com/bitnami/containers/issues
scdf-composed-task-runner 14:09:55.85
scdf-composed-task-runner 14:09:55.86 INFO ==> ** Starting Spring Cloud Dataflow Compose Task Runner **
_____ _ _______ _
/ ____| | | |__ __| | |
| | ___ _ __ ___ _ __ ___ ___ ___ __| | | | __ _ ___| | __
| | / _ \| '_ ` _ \| '_ \ / _ \/ __|/ _ \/ _` | | |/ _` / __| |/ /
| |___| (_) | | | | | | |_) | (_) \__ \ __/ (_| | | | (_| \__ \ <
\_____\___/|_| |_| |_| .__/ \___/|___/\___|\__,_| |_|\__,_|___/_|\_\
| |
|_|
_____
| __ \
| |__) | _ _ __ _ __ ___ _ __
| _ / | | | '_ \| '_ \ / _ \ '__|
| | \ \ |_| | | | | | | | __/ |
|_| \_\__,_|_| |_|_| |_|\___|_|
2023-11-09 14:10:00.556 INFO 1 --- [ main] o.s.c.d.c.ComposedTaskRunner : Starting ComposedTaskRunner using Java 17.0.8 on a1-l98r5r604l with PID 1 (/opt/bitnami/spring-cloud-dataflow-composed-task-runner/spring-cloud-dataflow-composed-task-runner-2.11.1.jar started by ? in /)
2023-11-09 14:10:00.559 INFO 1 --- [ main] o.s.c.d.c.ComposedTaskRunner : No active profile set, falling back to 1 default profile: "default"
2023-11-09 14:10:04.861 INFO 1 --- [ main] .s.d.r.c.RepositoryConfigurationDelegate : Multiple Spring Data modules found, entering strict repository configuration mode
2023-11-09 14:10:04.862 INFO 1 --- [ main] .s.d.r.c.RepositoryConfigurationDelegate : Bootstrapping Spring Data JPA repositories in DEFAULT mode.
2023-11-09 14:10:04.958 INFO 1 --- [ main] .s.d.r.c.RepositoryConfigurationDelegate : Finished Spring Data repository scanning in 11 ms. Found 0 JPA repository interfaces.
2023-11-09 14:10:06.657 INFO 1 --- [ main] trationDelegate$BeanPostProcessorChecker : Bean 'org.springframework.cloud.task.batch.configuration.TaskBatchAutoConfiguration' of type [org.springframework.cloud.task.batch.configuration.TaskBatchAutoConfiguration] is not eligible for getting processed by all BeanPostProcessors (for example: not eligible for auto-proxying)
2023-11-09 14:10:08.345 INFO 1 --- [ main] com.zaxxer.hikari.HikariDataSource : HikariPool-1 - Starting...
2023-11-09 14:10:09.053 INFO 1 --- [ main] com.zaxxer.hikari.HikariDataSource : HikariPool-1 - Start completed.
2023-11-09 14:10:09.653 INFO 1 --- [ main] o.hibernate.jpa.internal.util.LogHelper : HHH000204: Processing PersistenceUnitInfo [name: default]
2023-11-09 14:10:09.963 INFO 1 --- [ main] org.hibernate.Version : HHH000412: Hibernate ORM core version 5.6.15.Final
2023-11-09 14:10:10.753 INFO 1 --- [ main] o.hibernate.annotations.common.Version : HCANN000001: Hibernate Commons Annotations {5.1.2.Final}
2023-11-09 14:10:11.360 INFO 1 --- [ main] org.hibernate.dialect.Dialect : HHH000400: Using dialect: org.hibernate.dialect.MariaDB53Dialect
2023-11-09 14:10:12.665 INFO 1 --- [ main] o.h.e.t.j.p.i.JtaPlatformInitiator : HHH000490: Using JtaPlatform implementation: [org.hibernate.engine.transaction.jta.platform.internal.NoJtaPlatform]
2023-11-09 14:10:12.751 INFO 1 --- [ main] j.LocalContainerEntityManagerFactoryBean : Initialized JPA EntityManagerFactory for persistence unit 'default'
2023-11-09 14:10:13.371 INFO 1 --- [ main] .c.d.c.ComposedTaskStepExecutionListener : ComposedTaskStepExecutionListener supporting [a1-integration-task-batch, integration-task-batch, a1-task-alarm-composed, task-alarm-composed]
2023-11-09 14:10:13.460 INFO 1 --- [ main] o.s.b.c.r.s.JobRepositoryFactoryBean : No database type set, using meta data indicating: MYSQL
2023-11-09 14:10:13.572 INFO 1 --- [ main] o.s.b.c.r.s.JobRepositoryFactoryBean : No database type set, using meta data indicating: MYSQL
2023-11-09 14:10:13.651 INFO 1 --- [ main] o.s.b.c.l.support.SimpleJobLauncher : No TaskExecutor has been set, defaulting to synchronous executor.
2023-11-09 14:10:15.152 INFO 1 --- [ main] o.s.b.a.e.web.EndpointLinksResolver : Exposing 1 endpoint(s) beneath base path '/actuator'
2023-11-09 14:10:21.978 WARN 1 --- [ main] i.m.p.rsocket.PrometheusRSocketClient : Creating the connection and receiving the key timed out!
2023-11-09 14:10:22.059 WARN 1 --- [ main] onfigReactiveWebServerApplicationContext : Exception encountered during context initialization - cancelling refresh attempt: org.springframework.beans.factory.UnsatisfiedDependencyException: Error creating bean with name 'jobLauncherApplicationRunner': Unsatisfied dependency expressed through method 'setJobs' parameter 0; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'composedTaskJob': FactoryBean threw exception on object creation; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'a1-task-alarm-composed_0': FactoryBean threw exception on object creation; nested exception is java.lang.IllegalStateException: Duplicate key app.integration-task-batch.spring.cloud.task.tablePrefix (attempted merging values BOOT3_TASK_ and BOOT3_TASK_)
2023-11-09 14:10:22.061 INFO 1 --- [ main] j.LocalContainerEntityManagerFactoryBean : Closing JPA EntityManagerFactory for persistence unit 'default'
2023-11-09 14:10:22.144 INFO 1 --- [ main] com.zaxxer.hikari.HikariDataSource : HikariPool-1 - Shutdown initiated...
2023-11-09 14:10:22.147 INFO 1 --- [ main] com.zaxxer.hikari.HikariDataSource : HikariPool-1 - Shutdown completed.
2023-11-09 14:10:22.152 ERROR 1 --- [ main] o.s.c.t.listener.TaskLifecycleListener : An event to end a task has been received for a task that has not yet started.
2023-11-09 14:10:22.161 INFO 1 --- [ main] ConditionEvaluationReportLoggingListener :
Error starting ApplicationContext. To display the conditions report re-run your application with 'debug' enabled.
2023-11-09 14:10:22.262 ERROR 1 --- [ main] o.s.boot.SpringApplication : Application run failed
org.springframework.beans.factory.UnsatisfiedDependencyException: Error creating bean with name 'jobLauncherApplicationRunner': Unsatisfied dependency expressed through method 'setJobs' parameter 0; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'composedTaskJob': FactoryBean threw exception on object creation; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'a1-task-alarm-composed_0': FactoryBean threw exception on object creation; nested exception is java.lang.IllegalStateException: Duplicate key app.integration-task-batch.spring.cloud.task.tablePrefix (attempted merging values BOOT3_TASK_ and BOOT3_TASK_)
at org.springframework.beans.factory.annotation.AutowiredAnnotationBeanPostProcessor$AutowiredMethodElement.resolveMethodArguments(AutowiredAnnotationBeanPostProcessor.java:824) ~[spring-beans-5.3.30.jar!/:5.3.30]
at org.springframework.beans.factory.annotation.AutowiredAnnotationBeanPostProcessor$AutowiredMethodElement.inject(AutowiredAnnotationBeanPostProcessor.java:777) ~[spring-beans-5.3.30.jar!/:5.3.30]
at org.springframework.beans.factory.annotation.InjectionMetadata.inject(InjectionMetadata.java:119) ~[spring-beans-5.3.30.jar!/:5.3.30]
at org.springframework.beans.factory.annotation.AutowiredAnnotationBeanPostProcessor.postProcessProperties(AutowiredAnnotationBeanPostProcessor.java:408) ~[spring-beans-5.3.30.jar!/:5.3.30]
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.populateBean(AbstractAutowireCapableBeanFactory.java:1431) ~[spring-beans-5.3.30.jar!/:5.3.30]
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:619) ~[spring-beans-5.3.30.jar!/:5.3.30]
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:542) ~[spring-beans-5.3.30.jar!/:5.3.30]
at org.springframework.beans.factory.support.AbstractBeanFactory.lambda$doGetBean$0(AbstractBeanFactory.java:335) ~[spring-beans-5.3.30.jar!/:5.3.30]
at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:234) ~[spring-beans-5.3.30.jar!/:5.3.30]
at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:333) ~[spring-beans-5.3.30.jar!/:5.3.30]
at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:208) ~[spring-beans-5.3.30.jar!/:5.3.30]
at org.springframework.beans.factory.support.DefaultListableBeanFactory.preInstantiateSingletons(DefaultListableBeanFactory.java:955) ~[spring-beans-5.3.30.jar!/:5.3.30]
at org.springframework.context.support.AbstractApplicationContext.finishBeanFactoryInitialization(AbstractApplicationContext.java:921) ~[spring-context-5.3.30.jar!/:5.3.30]
at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:583) ~[spring-context-5.3.30.jar!/:5.3.30]
at org.springframework.boot.web.reactive.context.ReactiveWebServerApplicationContext.refresh(ReactiveWebServerApplicationContext.java:66) ~[spring-boot-2.7.16.jar!/:2.7.16]
at org.springframework.boot.SpringApplication.refresh(SpringApplication.java:731) ~[spring-boot-2.7.16.jar!/:2.7.16]
at org.springframework.boot.SpringApplication.refreshContext(SpringApplication.java:408) ~[spring-boot-2.7.16.jar!/:2.7.16]
at org.springframework.boot.SpringApplication.run(SpringApplication.java:307) ~[spring-boot-2.7.16.jar!/:2.7.16]
at org.springframework.boot.SpringApplication.run(SpringApplication.java:1303) ~[spring-boot-2.7.16.jar!/:2.7.16]
at org.springframework.boot.SpringApplication.run(SpringApplication.java:1292) ~[spring-boot-2.7.16.jar!/:2.7.16]
at org.springframework.cloud.dataflow.composedtaskrunner.ComposedTaskRunner.main(ComposedTaskRunner.java:31) ~[classes!/:na]
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:na]
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77) ~[na:na]
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:na]
at java.base/java.lang.reflect.Method.invoke(Method.java:568) ~[na:na]
at org.springframework.boot.loader.MainMethodRunner.run(MainMethodRunner.java:49) ~[spring-cloud-dataflow-composed-task-runner-2.11.1.jar:na]
at org.springframework.boot.loader.Launcher.launch(Launcher.java:108) ~[spring-cloud-dataflow-composed-task-runner-2.11.1.jar:na]
at org.springframework.boot.loader.Launcher.launch(Launcher.java:58) ~[spring-cloud-dataflow-composed-task-runner-2.11.1.jar:na]
at org.springframework.boot.loader.JarLauncher.main(JarLauncher.java:65) ~[spring-cloud-dataflow-composed-task-runner-2.11.1.jar:na]
Caused by: org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'composedTaskJob': FactoryBean threw exception on object creation; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'a1-task-alarm-composed_0': FactoryBean threw exception on object creation; nested exception is java.lang.IllegalStateException: Duplicate key app.integration-task-batch.spring.cloud.task.tablePrefix (attempted merging values BOOT3_TASK_ and BOOT3_TASK_)
at org.springframework.beans.factory.support.FactoryBeanRegistrySupport.doGetObjectFromFactoryBean(FactoryBeanRegistrySupport.java:176) ~[spring-beans-5.3.30.jar!/:5.3.30]
at org.springframework.beans.factory.support.FactoryBeanRegistrySupport.getObjectFromFactoryBean(FactoryBeanRegistrySupport.java:101) ~[spring-beans-5.3.30.jar!/:5.3.30]
at org.springframework.beans.factory.support.AbstractBeanFactory.getObjectForBeanInstance(AbstractBeanFactory.java:1898) ~[spring-beans-5.3.30.jar!/:5.3.30]
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.getObjectForBeanInstance(AbstractAutowireCapableBeanFactory.java:1284) ~[spring-beans-5.3.30.jar!/:5.3.30]
at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:267) ~[spring-beans-5.3.30.jar!/:5.3.30]
at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:208) ~[spring-beans-5.3.30.jar!/:5.3.30]
at org.springframework.beans.factory.config.DependencyDescriptor.resolveCandidate(DependencyDescriptor.java:276) ~[spring-beans-5.3.30.jar!/:5.3.30]
at org.springframework.beans.factory.support.DefaultListableBeanFactory.addCandidateEntry(DefaultListableBeanFactory.java:1609) ~[spring-beans-5.3.30.jar!/:5.3.30]
at org.springframework.beans.factory.support.DefaultListableBeanFactory.findAutowireCandidates(DefaultListableBeanFactory.java:1573) ~[spring-beans-5.3.30.jar!/:5.3.30]
at org.springframework.beans.factory.support.DefaultListableBeanFactory.resolveMultipleBeans(DefaultListableBeanFactory.java:1462) ~[spring-beans-5.3.30.jar!/:5.3.30]
at org.springframework.beans.factory.support.DefaultListableBeanFactory.doResolveDependency(DefaultListableBeanFactory.java:1349) ~[spring-beans-5.3.30.jar!/:5.3.30]
at org.springframework.beans.factory.support.DefaultListableBeanFactory.resolveDependency(DefaultListableBeanFactory.java:1311) ~[spring-beans-5.3.30.jar!/:5.3.30]
at org.springframework.beans.factory.annotation.AutowiredAnnotationBeanPostProcessor$AutowiredMethodElement.resolveMethodArguments(AutowiredAnnotationBeanPostProcessor.java:816) ~[spring-beans-5.3.30.jar!/:5.3.30]
... 28 common frames omitted
Caused by: org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'a1-task-alarm-composed_0': FactoryBean threw exception on object creation; nested exception is java.lang.IllegalStateException: Duplicate key app.integration-task-batch.spring.cloud.task.tablePrefix (attempted merging values BOOT3_TASK_ and BOOT3_TASK_)
at org.springframework.beans.factory.support.FactoryBeanRegistrySupport.doGetObjectFromFactoryBean(FactoryBeanRegistrySupport.java:176) ~[spring-beans-5.3.30.jar!/:5.3.30]
at org.springframework.beans.factory.support.FactoryBeanRegistrySupport.getObjectFromFactoryBean(FactoryBeanRegistrySupport.java:101) ~[spring-beans-5.3.30.jar!/:5.3.30]
at org.springframework.beans.factory.support.AbstractBeanFactory.getObjectForBeanInstance(AbstractBeanFactory.java:1898) ~[spring-beans-5.3.30.jar!/:5.3.30]
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.getObjectForBeanInstance(AbstractAutowireCapableBeanFactory.java:1284) ~[spring-beans-5.3.30.jar!/:5.3.30]
at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:267) ~[spring-beans-5.3.30.jar!/:5.3.30]
at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:213) ~[spring-beans-5.3.30.jar!/:5.3.30]
at org.springframework.context.support.AbstractApplicationContext.getBean(AbstractApplicationContext.java:1166) ~[spring-context-5.3.30.jar!/:5.3.30]
at org.springframework.cloud.dataflow.composedtaskrunner.ComposedRunnerJobFactory.getTaskAppFlow(ComposedRunnerJobFactory.java:359) ~[classes!/:na]
at org.springframework.cloud.dataflow.composedtaskrunner.ComposedRunnerJobFactory.createFlow(ComposedRunnerJobFactory.java:145) ~[classes!/:na]
at org.springframework.cloud.dataflow.composedtaskrunner.ComposedRunnerJobFactory.getObject(ComposedRunnerJobFactory.java:112) ~[classes!/:na]
at org.springframework.cloud.dataflow.composedtaskrunner.ComposedRunnerJobFactory.getObject(ComposedRunnerJobFactory.java:55) ~[classes!/:na]
at org.springframework.beans.factory.support.FactoryBeanRegistrySupport.doGetObjectFromFactoryBean(FactoryBeanRegistrySupport.java:169) ~[spring-beans-5.3.30.jar!/:5.3.30]
... 40 common frames omitted
Caused by: java.lang.IllegalStateException: Duplicate key app.integration-task-batch.spring.cloud.task.tablePrefix (attempted merging values BOOT3_TASK_ and BOOT3_TASK_)
at java.base/java.util.stream.Collectors.duplicateKeyException(Collectors.java:135) ~[na:na]
at java.base/java.util.stream.Collectors.lambda$uniqKeysMapAccumulator$1(Collectors.java:182) ~[na:na]
at java.base/java.util.stream.ReduceOps$3ReducingSink.accept(ReduceOps.java:169) ~[na:na]
at java.base/java.util.HashMap$EntrySpliterator.forEachRemaining(HashMap.java:1850) ~[na:na]
at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:509) ~[na:na]
at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:499) ~[na:na]
at java.base/java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:921) ~[na:na]
at java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234) ~[na:na]
at java.base/java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:682) ~[na:na]
at org.springframework.cloud.dataflow.core.Base64Utils.decodeMap(Base64Utils.java:62) ~[spring-cloud-dataflow-core-2.11.1.jar!/:na]
at org.springframework.cloud.dataflow.composedtaskrunner.ComposedTaskRunnerStepFactory.getObject(ComposedTaskRunnerStepFactory.java:157) ~[classes!/:na]
at org.springframework.cloud.dataflow.composedtaskrunner.ComposedTaskRunnerStepFactory.getObject(ComposedTaskRunnerStepFactory.java:59) ~[classes!/:na]
at org.springframework.beans.factory.support.FactoryBeanRegistrySupport.doGetObjectFromFactoryBean(FactoryBeanRegistrySupport.java:169) ~[spring-beans-5.3.30.jar!/:5.3.30]
... 51 common frames omittedArguments from failed runners pod:
Name: a1-l98r5r604l
Namespace: default
Priority: 0
Service Account: default
Node: ip-xx-xxx-xx-Xxx.ap-northeast-2.compute.internal/xx.xxx.xx.xxx
Start Time: Thu, 09 Nov 2023 23:09:55 +0900
Labels: role=spring-app
spring-app-id=a1-l98r5r604l
spring-deployment-id=a1-l98r5r604l
task-name=a1
Annotations: kubernetes.io/psp: eks.privileged
Status: Failed
IP: xx.xxx.xx.xxx
IPs:
IP: xx.xxx.xx.xxx
Containers:
a1-gdg6z6el8o:
Container ID: containerd://de974d308b0dce61d9ac6e7e07734f4033e49da1427c5c8fbca26e0f53a4b1c9
Image: docker.io/bitnami/spring-cloud-dataflow-composed-task-runner:2.11.1-debian-11-r0
Image ID: docker.io/bitnami/spring-cloud-dataflow-composed-task-runner@sha256:2495dcbebd784314e45b431760fe61c375348b906a6c451bdb809b825049de18
Port: <none>
Host Port: <none>
Args:
...
--spring.cloud.task.name=a1
--composed-task-properties=version.a1-integration-task-batch.integration-task-batch=latest
--management.metrics.tags.service=task-application
--composed-task-app-properties.base64_<REDACTED>=boot3
--composed-task-app-properties.base64_<REDACTED>=Always
--composed-task-app-properties.base64_<REDACTED>=3
--spring.cloud.task.initialize-enabled=false
--spring.cloud.task.tablePrefix=TASK_
--composed-task-app-properties.base64_<REDACTED>=boot3
--composed-task-app-properties.base64_<REDACTED>=BOOT3_BATCH_
--composed-task-app-properties.base64_<REDACTED>=3
--management.metrics.tags.application=${spring.cloud.task.name:unknown}-${spring.cloud.task.executionid:unknown}
--composed-task-app-properties.base64_<REDACTED>=BOOT3_TASK_
--composed-task-app-properties.base64_<REDACTED>=BOOT3_TASK_
--composed-task-app-properties.base64_<REDACTED>=3
--platform-name=default
--composed-task-app-properties.app.a1-task-alarm-composed.spring.cloud.task.tablePrefix=BOOT3_TASK_
--graph=a1-integration-task-batch && a1-task-alarm-composed
--spring.datasource.password=<REDACTED>
--spring.cloud.deployer.bootVersion=2
...
--spring.batch.jdbc.table-prefix=BATCH_
--spring.cloud.task.schemaTarget=boot2
--composed-task-app-properties.app.integration-task-batch.spring.cloud.task.tablePrefix=BOOT3_TASK_
--spring.cloud.task.executionid=106
--spring.cloud.data.flow.taskappname=composed-task-runner
State: Terminated
Reason: Error
Exit Code: 1
Started: Thu, 09 Nov 2023 23:09:55 +0900
Finished: Thu, 09 Nov 2023 23:10:22 +0900
Ready: False
Restart Count: 0
Limits:
cpu: 500m
memory: 1Gi
Requests:
cpu: 500m
memory: 1GiEven though it has been deleted from the Web UI and ConfigMap, the related arguments remain in CTR(Composed Task Runner).
How can I launch CTR without the following properties?
--spring.cloud.task.schemaTarget=boot3
--spring.cloud.task.tablePrefix
--spring.cloud.deployer.bootVersion=3Delete the previous task executions and then relaunch.
Dataflow does not currently allow the user to delete previously used properties or args.
younsl
commented
1 year ago It's not worked for me.
There is a last execution:
As you mentioned, Click the [Clean up] button to delete all last executions:
Launch task without the following properties:
spring.cloud.deployer.bootVersion = 3spring.cloud.task.tablePrefix = BOOT3_TASK_spring.cloud.task.schemaTarget = boot3Failed pod logs:
scdf-composed-task-runner 00:51:12.57
scdf-composed-task-runner 00:51:12.57 Welcome to the Bitnami spring-cloud-dataflow-composed-task-runner container
scdf-composed-task-runner 00:51:12.57 Subscribe to project updates by watching https://github.com/bitnami/containers
scdf-composed-task-runner 00:51:12.57 Submit issues and feature requests at https://github.com/bitnami/containers/issues
scdf-composed-task-runner 00:51:12.58
scdf-composed-task-runner 00:51:12.58 INFO ==> ** Starting Spring Cloud Dataflow Compose Task Runner **
_____ _ _______ _
/ ____| | | |__ __| | |
| | ___ _ __ ___ _ __ ___ ___ ___ __| | | | __ _ ___| | __
| | / _ \| '_ ` _ \| '_ \ / _ \/ __|/ _ \/ _` | | |/ _` / __| |/ /
| |___| (_) | | | | | | |_) | (_) \__ \ __/ (_| | | | (_| \__ \ <
\_____\___/|_| |_| |_| .__/ \___/|___/\___|\__,_| |_|\__,_|___/_|\_\
| |
|_|
_____
| __ \
| |__) | _ _ __ _ __ ___ _ __
| _ / | | | '_ \| '_ \ / _ \ '__|
| | \ \ |_| | | | | | | | __/ |
|_| \_\__,_|_| |_|_| |_|\___|_|
2023-11-10 00:51:17.059 INFO 1 --- [ main] o.s.c.d.c.ComposedTaskRunner : Starting ComposedTaskRunner using Java 17.0.8 on a1-kz5m9791k7 with PID 1 (/opt/bitnami/spring-cloud-dataflow-composed-task-runner/spring-cloud-dataflow-composed-task-runner-2.11.1.jar started by ? in /)
2023-11-10 00:51:17.061 INFO 1 --- [ main] o.s.c.d.c.ComposedTaskRunner : No active profile set, falling back to 1 default profile: "default"
2023-11-10 00:51:21.348 INFO 1 --- [ main] .s.d.r.c.RepositoryConfigurationDelegate : Multiple Spring Data modules found, entering strict repository configuration mode
2023-11-10 00:51:21.350 INFO 1 --- [ main] .s.d.r.c.RepositoryConfigurationDelegate : Bootstrapping Spring Data JPA repositories in DEFAULT mode.
2023-11-10 00:51:21.449 INFO 1 --- [ main] .s.d.r.c.RepositoryConfigurationDelegate : Finished Spring Data repository scanning in 91 ms. Found 0 JPA repository interfaces.
2023-11-10 00:51:23.259 INFO 1 --- [ main] trationDelegate$BeanPostProcessorChecker : Bean 'org.springframework.cloud.task.batch.configuration.TaskBatchAutoConfiguration' of type [org.springframework.cloud.task.batch.configuration.TaskBatchAutoConfiguration] is not eligible for getting processed by all BeanPostProcessors (for example: not eligible for auto-proxying)
2023-11-10 00:51:24.955 INFO 1 --- [ main] com.zaxxer.hikari.HikariDataSource : HikariPool-1 - Starting...
2023-11-10 00:51:25.653 INFO 1 --- [ main] com.zaxxer.hikari.HikariDataSource : HikariPool-1 - Start completed.
2023-11-10 00:51:26.356 INFO 1 --- [ main] o.hibernate.jpa.internal.util.LogHelper : HHH000204: Processing PersistenceUnitInfo [name: default]
2023-11-10 00:51:26.746 INFO 1 --- [ main] org.hibernate.Version : HHH000412: Hibernate ORM core version 5.6.15.Final
2023-11-10 00:51:27.643 INFO 1 --- [ main] o.hibernate.annotations.common.Version : HCANN000001: Hibernate Commons Annotations {5.1.2.Final}
2023-11-10 00:51:28.260 INFO 1 --- [ main] org.hibernate.dialect.Dialect : HHH000400: Using dialect: org.hibernate.dialect.MariaDB53Dialect
2023-11-10 00:51:29.742 INFO 1 --- [ main] o.h.e.t.j.p.i.JtaPlatformInitiator : HHH000490: Using JtaPlatform implementation: [org.hibernate.engine.transaction.jta.platform.internal.NoJtaPlatform]
2023-11-10 00:51:29.842 INFO 1 --- [ main] j.LocalContainerEntityManagerFactoryBean : Initialized JPA EntityManagerFactory for persistence unit 'default'
2023-11-10 00:51:30.681 INFO 1 --- [ main] .c.d.c.ComposedTaskStepExecutionListener : ComposedTaskStepExecutionListener supporting [a1-integration-task-batch, integration-task-batch, a1-task-alarm-composed, task-alarm-composed]
2023-11-10 00:51:30.850 INFO 1 --- [ main] o.s.b.c.r.s.JobRepositoryFactoryBean : No database type set, using meta data indicating: MYSQL
2023-11-10 00:51:30.956 INFO 1 --- [ main] o.s.b.c.r.s.JobRepositoryFactoryBean : No database type set, using meta data indicating: MYSQL
2023-11-10 00:51:30.971 INFO 1 --- [ main] o.s.b.c.l.support.SimpleJobLauncher : No TaskExecutor has been set, defaulting to synchronous executor.
2023-11-10 00:51:32.658 INFO 1 --- [ main] o.s.b.a.e.web.EndpointLinksResolver : Exposing 1 endpoint(s) beneath base path '/actuator'
2023-11-10 00:51:39.655 WARN 1 --- [ main] i.m.p.rsocket.PrometheusRSocketClient : Creating the connection and receiving the key timed out!
2023-11-10 00:51:40.051 INFO 1 --- [ main] ctiveUserDetailsServiceAutoConfiguration :
2023-11-10 00:51:40.859 INFO 1 --- [ main] o.s.c.d.c.ComposedTaskRunnerTaskListener : onTaskStartup:executionId=110
2023-11-10 00:51:40.943 INFO 1 --- [ main] o.s.b.web.embedded.netty.NettyWebServer : Netty started on port 8080
2023-11-10 00:51:40.959 INFO 1 --- [ main] o.s.c.d.c.ComposedTaskRunner : Started ComposedTaskRunner in 26.502 seconds (JVM running for 28.365)
2023-11-10 00:51:41.043 INFO 1 --- [ main] o.s.b.a.b.JobLauncherApplicationRunner : Running default command line with: []
2023-11-10 00:51:41.043 INFO 1 --- [ main] o.s.b.c.r.s.JobRepositoryFactoryBean : No database type set, using meta data indicating: MYSQL
2023-11-10 00:51:41.246 INFO 1 --- [ main] o.s.b.c.l.support.SimpleJobLauncher : Job: [FlowJob: [name=a1]] launched with the following parameters: [{run.id=1}]
2023-11-10 00:51:41.255 INFO 1 --- [ main] o.s.c.t.b.l.TaskBatchExecutionListener : The job execution id 37 was run within the task execution 110
2023-11-10 00:51:41.345 INFO 1 --- [ main] o.s.batch.core.job.SimpleStepHandler : Executing step: [a1-integration-task-batch_0]
2023-11-10 00:51:41.350 INFO 1 --- [ main] .c.d.c.ComposedTaskStepExecutionListener : beforeStep:a1-integration-task-batch_0:37>>>>
2023-11-10 00:51:41.351 INFO 1 --- [ main] .c.d.c.ComposedTaskStepExecutionListener : beforeStep:a1-integration-task-batch_0:37<<<
2023-11-10 00:51:41.950 ERROR 1 --- [ main] o.s.batch.core.step.AbstractStep : Encountered an error executing step a1-integration-task-batch_0 in job a1
org.springframework.web.client.HttpClientErrorException$Unauthorized: 401 : [no body]
at org.springframework.web.client.HttpClientErrorException.create(HttpClientErrorException.java:105) ~[spring-web-5.3.30.jar!/:5.3.30]
at org.springframework.web.client.DefaultResponseErrorHandler.handleError(DefaultResponseErrorHandler.java:168) ~[spring-web-5.3.30.jar!/:5.3.30]
at org.springframework.web.client.DefaultResponseErrorHandler.handleError(DefaultResponseErrorHandler.java:122) ~[spring-web-5.3.30.jar!/:5.3.30]
at org.springframework.cloud.dataflow.rest.client.VndErrorResponseErrorHandler.handleError(VndErrorResponseErrorHandler.java:62) ~[spring-cloud-dataflow-rest-client-2.11.1.jar!/:na]
at org.springframework.web.client.ResponseErrorHandler.handleError(ResponseErrorHandler.java:63) ~[spring-web-5.3.30.jar!/:5.3.30]
at org.springframework.web.client.RestTemplate.handleResponse(RestTemplate.java:825) ~[spring-web-5.3.30.jar!/:5.3.30]
at org.springframework.web.client.RestTemplate.doExecute(RestTemplate.java:783) ~[spring-web-5.3.30.jar!/:5.3.30]
at org.springframework.web.client.RestTemplate.execute(RestTemplate.java:757) ~[spring-web-5.3.30.jar!/:5.3.30]
at org.springframework.web.client.RestTemplate.getForObject(RestTemplate.java:358) ~[spring-web-5.3.30.jar!/:5.3.30]
at org.springframework.cloud.dataflow.rest.client.DataFlowTemplate.<init>(DataFlowTemplate.java:150) ~[spring-cloud-dataflow-rest-client-2.11.1.jar!/:na]
at org.springframework.cloud.dataflow.composedtaskrunner.TaskLauncherTasklet.dataFlowOperations(TaskLauncherTasklet.java:327) ~[classes!/:na]
at org.springframework.cloud.dataflow.composedtaskrunner.TaskLauncherTasklet.taskOperations(TaskLauncherTasklet.java:265) ~[classes!/:na]
at org.springframework.cloud.dataflow.composedtaskrunner.TaskLauncherTasklet.execute(TaskLauncherTasklet.java:163) ~[classes!/:na]
at org.springframework.batch.core.step.tasklet.TaskletStep$ChunkTransactionCallback.doInTransaction(TaskletStep.java:407) ~[spring-batch-core-4.3.9.jar!/:4.3.9]
at org.springframework.batch.core.step.tasklet.TaskletStep$ChunkTransactionCallback.doInTransaction(TaskletStep.java:331) ~[spring-batch-core-4.3.9.jar!/:4.3.9]
at org.springframework.transaction.support.TransactionTemplate.execute(TransactionTemplate.java:140) ~[spring-tx-5.3.30.jar!/:5.3.30]
at org.springframework.batch.core.step.tasklet.TaskletStep$2.doInChunkContext(TaskletStep.java:273) ~[spring-batch-core-4.3.9.jar!/:4.3.9]
at org.springframework.batch.core.scope.context.StepContextRepeatCallback.doInIteration(StepContextRepeatCallback.java:82) ~[spring-batch-core-4.3.9.jar!/:4.3.9]
at org.springframework.batch.repeat.support.RepeatTemplate.getNextResult(RepeatTemplate.java:375) ~[spring-batch-infrastructure-4.3.9.jar!/:4.3.9]
at org.springframework.batch.repeat.support.RepeatTemplate.executeInternal(RepeatTemplate.java:215) ~[spring-batch-infrastructure-4.3.9.jar!/:4.3.9]
at org.springframework.batch.repeat.support.RepeatTemplate.iterate(RepeatTemplate.java:145) ~[spring-batch-infrastructure-4.3.9.jar!/:4.3.9]
at org.springframework.batch.core.step.tasklet.TaskletStep.doExecute(TaskletStep.java:258) ~[spring-batch-core-4.3.9.jar!/:4.3.9]
at org.springframework.batch.core.step.AbstractStep.execute(AbstractStep.java:208) ~[spring-batch-core-4.3.9.jar!/:4.3.9]
at org.springframework.batch.core.job.SimpleStepHandler.handleStep(SimpleStepHandler.java:152) ~[spring-batch-core-4.3.9.jar!/:4.3.9]
at org.springframework.batch.core.job.flow.JobFlowExecutor.executeStep(JobFlowExecutor.java:68) ~[spring-batch-core-4.3.9.jar!/:4.3.9]
at org.springframework.batch.core.job.flow.support.state.StepState.handle(StepState.java:68) ~[spring-batch-core-4.3.9.jar!/:4.3.9]
at org.springframework.batch.core.job.flow.support.SimpleFlow.resume(SimpleFlow.java:167) ~[spring-batch-core-4.3.9.jar!/:4.3.9]
at org.springframework.batch.core.job.flow.support.SimpleFlow.start(SimpleFlow.java:142) ~[spring-batch-core-4.3.9.jar!/:4.3.9]
at org.springframework.batch.core.job.flow.support.state.FlowState.handle(FlowState.java:56) ~[spring-batch-core-4.3.9.jar!/:4.3.9]
at org.springframework.batch.core.job.flow.support.SimpleFlow.resume(SimpleFlow.java:167) ~[spring-batch-core-4.3.9.jar!/:4.3.9]
at org.springframework.batch.core.job.flow.support.SimpleFlow.start(SimpleFlow.java:142) ~[spring-batch-core-4.3.9.jar!/:4.3.9]
at org.springframework.batch.core.job.flow.FlowJob.doExecute(FlowJob.java:139) ~[spring-batch-core-4.3.9.jar!/:4.3.9]
at org.springframework.batch.core.job.AbstractJob.execute(AbstractJob.java:320) ~[spring-batch-core-4.3.9.jar!/:4.3.9]
at org.springframework.batch.core.launch.support.SimpleJobLauncher$1.run(SimpleJobLauncher.java:149) ~[spring-batch-core-4.3.9.jar!/:4.3.9]
at org.springframework.core.task.SyncTaskExecutor.execute(SyncTaskExecutor.java:50) ~[spring-core-5.3.30.jar!/:5.3.30]
at org.springframework.batch.core.launch.support.SimpleJobLauncher.run(SimpleJobLauncher.java:140) ~[spring-batch-core-4.3.9.jar!/:4.3.9]
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:na]
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77) ~[na:na]
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:na]
at java.base/java.lang.reflect.Method.invoke(Method.java:568) ~[na:na]
at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:344) ~[spring-aop-5.3.30.jar!/:5.3.30]
at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:198) ~[spring-aop-5.3.30.jar!/:5.3.30]
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163) ~[spring-aop-5.3.30.jar!/:5.3.30]
at org.springframework.batch.core.configuration.annotation.SimpleBatchConfiguration$PassthruAdvice.invoke(SimpleBatchConfiguration.java:128) ~[spring-batch-core-4.3.9.jar!/:4.3.9]
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) ~[spring-aop-5.3.30.jar!/:5.3.30]
at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:241) ~[spring-aop-5.3.30.jar!/:5.3.30]
at jdk.proxy2/jdk.proxy2.$Proxy92.run(Unknown Source) ~[na:na]
at org.springframework.boot.autoconfigure.batch.JobLauncherApplicationRunner.execute(JobLauncherApplicationRunner.java:213) ~[spring-boot-autoconfigure-2.7.16.jar!/:2.7.16]
at org.springframework.boot.autoconfigure.batch.JobLauncherApplicationRunner.executeLocalJobs(JobLauncherApplicationRunner.java:193) ~[spring-boot-autoconfigure-2.7.16.jar!/:2.7.16]
at org.springframework.boot.autoconfigure.batch.JobLauncherApplicationRunner.launchJobFromProperties(JobLauncherApplicationRunner.java:172) ~[spring-boot-autoconfigure-2.7.16.jar!/:2.7.16]
at org.springframework.boot.autoconfigure.batch.JobLauncherApplicationRunner.run(JobLauncherApplicationRunner.java:167) ~[spring-boot-autoconfigure-2.7.16.jar!/:2.7.16]
at org.springframework.boot.autoconfigure.batch.JobLauncherApplicationRunner.run(JobLauncherApplicationRunner.java:162) ~[spring-boot-autoconfigure-2.7.16.jar!/:2.7.16]
at org.springframework.boot.SpringApplication.callRunner(SpringApplication.java:759) ~[spring-boot-2.7.16.jar!/:2.7.16]
at org.springframework.boot.SpringApplication.callRunners(SpringApplication.java:749) ~[spring-boot-2.7.16.jar!/:2.7.16]
at org.springframework.boot.SpringApplication.run(SpringApplication.java:314) ~[spring-boot-2.7.16.jar!/:2.7.16]
at org.springframework.boot.SpringApplication.run(SpringApplication.java:1303) ~[spring-boot-2.7.16.jar!/:2.7.16]
at org.springframework.boot.SpringApplication.run(SpringApplication.java:1292) ~[spring-boot-2.7.16.jar!/:2.7.16]
at org.springframework.cloud.dataflow.composedtaskrunner.ComposedTaskRunner.main(ComposedTaskRunner.java:31) ~[classes!/:na]
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:na]
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77) ~[na:na]
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:na]
at java.base/java.lang.reflect.Method.invoke(Method.java:568) ~[na:na]
at org.springframework.boot.loader.MainMethodRunner.run(MainMethodRunner.java:49) ~[spring-cloud-dataflow-composed-task-runner-2.11.1.jar:na]
at org.springframework.boot.loader.Launcher.launch(Launcher.java:108) ~[spring-cloud-dataflow-composed-task-runner-2.11.1.jar:na]
at org.springframework.boot.loader.Launcher.launch(Launcher.java:58) ~[spring-cloud-dataflow-composed-task-runner-2.11.1.jar:na]
at org.springframework.boot.loader.JarLauncher.main(JarLauncher.java:65) ~[spring-cloud-dataflow-composed-task-runner-2.11.1.jar:na]
2023-11-10 00:51:41.951 INFO 1 --- [ main] .c.d.c.ComposedTaskStepExecutionListener : AfterStep processing for stepExecution a1-integration-task-batch_0:37
2023-11-10 00:51:41.951 ERROR 1 --- [ main] o.s.batch.core.step.AbstractStep : Exception in afterStep callback in step a1-integration-task-batch_0 in job a1
java.lang.IllegalArgumentException: TaskLauncherTasklet for job 37 did not return a task-execution-id. Check to see if task exists.
at org.springframework.util.Assert.notNull(Assert.java:201) ~[spring-core-5.3.30.jar!/:5.3.30]
at org.springframework.cloud.dataflow.composedtaskrunner.ComposedTaskStepExecutionListener.afterStep(ComposedTaskStepExecutionListener.java:67) ~[classes!/:na]
at org.springframework.batch.core.listener.CompositeStepExecutionListener.afterStep(CompositeStepExecutionListener.java:64) ~[spring-batch-core-4.3.9.jar!/:4.3.9]
at org.springframework.batch.core.step.AbstractStep.execute(AbstractStep.java:247) ~[spring-batch-core-4.3.9.jar!/:4.3.9]
at org.springframework.batch.core.job.SimpleStepHandler.handleStep(SimpleStepHandler.java:152) ~[spring-batch-core-4.3.9.jar!/:4.3.9]
at org.springframework.batch.core.job.flow.JobFlowExecutor.executeStep(JobFlowExecutor.java:68) ~[spring-batch-core-4.3.9.jar!/:4.3.9]
at org.springframework.batch.core.job.flow.support.state.StepState.handle(StepState.java:68) ~[spring-batch-core-4.3.9.jar!/:4.3.9]
at org.springframework.batch.core.job.flow.support.SimpleFlow.resume(SimpleFlow.java:167) ~[spring-batch-core-4.3.9.jar!/:4.3.9]
at org.springframework.batch.core.job.flow.support.SimpleFlow.start(SimpleFlow.java:142) ~[spring-batch-core-4.3.9.jar!/:4.3.9]
at org.springframework.batch.core.job.flow.support.state.FlowState.handle(FlowState.java:56) ~[spring-batch-core-4.3.9.jar!/:4.3.9]
at org.springframework.batch.core.job.flow.support.SimpleFlow.resume(SimpleFlow.java:167) ~[spring-batch-core-4.3.9.jar!/:4.3.9]
at org.springframework.batch.core.job.flow.support.SimpleFlow.start(SimpleFlow.java:142) ~[spring-batch-core-4.3.9.jar!/:4.3.9]
at org.springframework.batch.core.job.flow.FlowJob.doExecute(FlowJob.java:139) ~[spring-batch-core-4.3.9.jar!/:4.3.9]
at org.springframework.batch.core.job.AbstractJob.execute(AbstractJob.java:320) ~[spring-batch-core-4.3.9.jar!/:4.3.9]
at org.springframework.batch.core.launch.support.SimpleJobLauncher$1.run(SimpleJobLauncher.java:149) ~[spring-batch-core-4.3.9.jar!/:4.3.9]
at org.springframework.core.task.SyncTaskExecutor.execute(SyncTaskExecutor.java:50) ~[spring-core-5.3.30.jar!/:5.3.30]
at org.springframework.batch.core.launch.support.SimpleJobLauncher.run(SimpleJobLauncher.java:140) ~[spring-batch-core-4.3.9.jar!/:4.3.9]
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:na]
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77) ~[na:na]
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:na]
at java.base/java.lang.reflect.Method.invoke(Method.java:568) ~[na:na]
at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:344) ~[spring-aop-5.3.30.jar!/:5.3.30]
at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:198) ~[spring-aop-5.3.30.jar!/:5.3.30]
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163) ~[spring-aop-5.3.30.jar!/:5.3.30]
at org.springframework.batch.core.configuration.annotation.SimpleBatchConfiguration$PassthruAdvice.invoke(SimpleBatchConfiguration.java:128) ~[spring-batch-core-4.3.9.jar!/:4.3.9]
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) ~[spring-aop-5.3.30.jar!/:5.3.30]
at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:241) ~[spring-aop-5.3.30.jar!/:5.3.30]
at jdk.proxy2/jdk.proxy2.$Proxy92.run(Unknown Source) ~[na:na]
at org.springframework.boot.autoconfigure.batch.JobLauncherApplicationRunner.execute(JobLauncherApplicationRunner.java:213) ~[spring-boot-autoconfigure-2.7.16.jar!/:2.7.16]
at org.springframework.boot.autoconfigure.batch.JobLauncherApplicationRunner.executeLocalJobs(JobLauncherApplicationRunner.java:193) ~[spring-boot-autoconfigure-2.7.16.jar!/:2.7.16]
at org.springframework.boot.autoconfigure.batch.JobLauncherApplicationRunner.launchJobFromProperties(JobLauncherApplicationRunner.java:172) ~[spring-boot-autoconfigure-2.7.16.jar!/:2.7.16]
at org.springframework.boot.autoconfigure.batch.JobLauncherApplicationRunner.run(JobLauncherApplicationRunner.java:167) ~[spring-boot-autoconfigure-2.7.16.jar!/:2.7.16]
at org.springframework.boot.autoconfigure.batch.JobLauncherApplicationRunner.run(JobLauncherApplicationRunner.java:162) ~[spring-boot-autoconfigure-2.7.16.jar!/:2.7.16]
at org.springframework.boot.SpringApplication.callRunner(SpringApplication.java:759) ~[spring-boot-2.7.16.jar!/:2.7.16]
at org.springframework.boot.SpringApplication.callRunners(SpringApplication.java:749) ~[spring-boot-2.7.16.jar!/:2.7.16]
at org.springframework.boot.SpringApplication.run(SpringApplication.java:314) ~[spring-boot-2.7.16.jar!/:2.7.16]
at org.springframework.boot.SpringApplication.run(SpringApplication.java:1303) ~[spring-boot-2.7.16.jar!/:2.7.16]
at org.springframework.boot.SpringApplication.run(SpringApplication.java:1292) ~[spring-boot-2.7.16.jar!/:2.7.16]
at org.springframework.cloud.dataflow.composedtaskrunner.ComposedTaskRunner.main(ComposedTaskRunner.java:31) ~[classes!/:na]
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:na]
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77) ~[na:na]
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:na]
at java.base/java.lang.reflect.Method.invoke(Method.java:568) ~[na:na]
at org.springframework.boot.loader.MainMethodRunner.run(MainMethodRunner.java:49) ~[spring-cloud-dataflow-composed-task-runner-2.11.1.jar:na]
at org.springframework.boot.loader.Launcher.launch(Launcher.java:108) ~[spring-cloud-dataflow-composed-task-runner-2.11.1.jar:na]
at org.springframework.boot.loader.Launcher.launch(Launcher.java:58) ~[spring-cloud-dataflow-composed-task-runner-2.11.1.jar:na]
at org.springframework.boot.loader.JarLauncher.main(JarLauncher.java:65) ~[spring-cloud-dataflow-composed-task-runner-2.11.1.jar:na]
2023-11-10 00:51:41.954 INFO 1 --- [ main] o.s.batch.core.step.AbstractStep : Step: [a1-integration-task-batch_0] executed in 608ms
2023-11-10 00:51:41.968 INFO 1 --- [ main] o.s.b.c.l.support.SimpleJobLauncher : Job: [FlowJob: [name=a1]] completed with the following parameters: [{run.id=1}] and the following status: [FAILED] in 716ms
2023-11-10 00:51:42.057 INFO 1 --- [ main] j.LocalContainerEntityManagerFactoryBean : Closing JPA EntityManagerFactory for persistence unit 'default'
2023-11-10 00:51:42.059 INFO 1 --- [ main] com.zaxxer.hikari.HikariDataSource : HikariPool-1 - Shutdown initiated...
2023-11-10 00:51:42.063 INFO 1 --- [ main] com.zaxxer.hikari.HikariDataSource : HikariPool-1 - Shutdown completed.The failed pod returns error message org.springframework.web.client.HttpClientErrorException$Unauthorized: 401 : [no body].
Task executions info:
There are boot2 arguments that I have never set.
younsl
commented
1 year ago Is there any further progress or additional solutions to try?
cppwfs
commented
1 year ago This is a different issue. Before the Composed Task Runner was failing. In this case, it looks like the a1-integration-task-batch_0 did not run. Can you check the log of this app and share its results?
younsl
commented
1 year ago As you mentioned, I ran individual task a1-integration-task-batch_0 rather than Composed Task.
Results:
Individual task a1-integration-task-batch was completed without problem by boot3 version.
When executing one task, the Schema Target value and Arguments are set to boot3 without any special settings.
Task execution details:
Full logs for the executed pod:
dev
OpenJDK 64-Bit Server VM warning: Sharing is only supported for boot loader classes because bootstrap classpath has been appended
OpenJDK 64-Bit Server VM warning: Sharing is only supported for boot loader classes because bootstrap classpath has been appended
OpenJDK 64-Bit Server VM warning: Sharing is only supported for boot loader classes because bootstrap classpath has been appended
OpenJDK 64-Bit Server VM warning: Sharing is only supported for boot loader classes because bootstrap classpath has been appended
2023-11-14T00:56:42,498+0000 [7 1] com.newrelic INFO: New Relic Agent: Loading configuration file "./newrelic.yml"
2023-11-14T00:56:42,610+0000 [7 1] com.newrelic INFO: Using default collector host: collector.newrelic.com
2023-11-14T00:56:42,685+0000 [7 1] com.newrelic INFO: Using default metric ingest URI: https://metric-api.newrelic.com/metric/v1
2023-11-14T00:56:42,685+0000 [7 1] com.newrelic INFO: Using default event ingest URI: https://insights-collector.newrelic.com/v1/accounts/events
2023-11-14T00:56:42,891+0000 [7 1] com.newrelic INFO: New Relic Agent: Writing to log file: /opt/newrelic/logs/newrelic_agent.log
OpenJDK 64-Bit Server VM warning: Sharing is only supported for boot loader classes because bootstrap classpath has been appended
OpenJDK 64-Bit Server VM warning: Sharing is only supported for boot loader classes because bootstrap classpath has been appended
OpenJDK 64-Bit Server VM warning: Sharing is only supported for boot loader classes because bootstrap classpath has been appended
OpenJDK 64-Bit Server VM warning: Sharing is only supported for boot loader classes because bootstrap classpath has been appended
. ____ _ __ _ _
/\\ / ___'_ __ _ _(_)_ __ __ _ \ \ \ \
( ( )\___ | '_ | '_| | '_ \/ _` | \ \ \ \
\\/ ___)| |_)| | | | | || (_| | ) ) ) )
' |____| .__|_| |_|_| |_\__, | / / / /
=========|_|==============|___/=/_/_/_/
:: Spring Boot :: (v3.1.4)
2023-11-14T00:57:14.001Z INFO 7 --- [ main] c.c.c.ConfigServicePropertySourceLocator : Fetching config from server at : http://<REDACTED>
2023-11-14T00:57:17.601Z INFO 7 --- [ main] c.c.c.ConfigServicePropertySourceLocator : Located environment: name=TRADE-INTEGRATION-BATCH, profiles=[dev], label=null, version=null, state=null
2023-11-14T00:57:17.602Z INFO 7 --- [ main] b.c.PropertySourceBootstrapConfiguration : Located property source: ...
2023-11-14T00:57:17.699Z INFO 7 --- [ main] c.i.IntegrationApplication$Companion : The following 1 profile is active: "dev"
2023-11-14T00:57:20.892Z TRACE 7 --- [ main] o.s.c.t.b.c.JobLaunchCondition : Condition JobLaunchCondition on org.springframework.cloud.task.batch.configuration.TaskJobLauncherAutoConfiguration did not match due to AllNestedConditions 0 matched 2 did not; NestedCondition on JobLaunchCondition.SpringBatchJobCondition @ConditionalOnProperty (spring.batch.job.
2023-11-14T00:57:29.003Z INFO 7 --- [ main] o.s.cloud.context.scope.GenericScope : BeanFactory id=315efcec-bc95-3f5e-92cb-ec7cac7e1379
2023-11-14T00:57:33.798Z INFO 7 --- [ main] o.s.b.w.embedded.tomcat.TomcatWebServer : Tomcat initialized with port(s): 8080 (http)
2023-11-14T00:57:33.891Z INFO 7 --- [ main] o.apache.catalina.core.StandardService : Starting service [Tomcat]
2023-11-14T00:57:33.891Z INFO 7 --- [ main] o.apache.catalina.core.StandardEngine : Starting Servlet engine: [Apache Tomcat/10.1.13]
2023-11-14T00:57:34.306Z INFO 7 --- [ main] o.a.c.c.C.[Tomcat].[localhost].[/] : Initializing Spring embedded WebApplicationContext
2023-11-14T00:57:34.306Z INFO 7 --- [ main] w.s.c.ServletWebServerApplicationContext : Root WebApplicationContext: initialization completed in 16418 ms
2023-11-14T00:57:38.090Z INFO 7 --- [ main] com.zaxxer.hikari.HikariDataSource : HikariPool-1 - Starting...
2023-11-14T00:57:40.789Z INFO 7 --- [ main] com.zaxxer.hikari.pool.HikariPool : HikariPool-1 - Added connection com.mysql.cj.jdbc.ConnectionImpl@15c8b75c
2023-11-14T00:57:40.792Z INFO 7 --- [ main] com.zaxxer.hikari.HikariDataSource : HikariPool-1 - Start completed.
2023-11-14T00:57:41.094Z DEBUG 7 --- [ main] o.s.c.t.c.SimpleTaskAutoConfiguration : Using xxxxxx.integration.config.task.TaskConfiguration$$SpringCGLIB$$0 TaskConfigurer
2023-11-14T00:57:41.095Z DEBUG 7 --- [ main] o.s.c.t.c.DefaultTaskConfigurer : No EntityManager was found, using DataSourceTransactionManager
2023-11-14T00:57:43.190Z INFO 7 --- [ main] com.zaxxer.hikari.HikariDataSource : HikariPool-2 - Starting...
2023-11-14T00:57:43.288Z INFO 7 --- [ main] com.zaxxer.hikari.pool.HikariPool : HikariPool-2 - Added connection com.mysql.cj.jdbc.ConnectionImpl@27558dee
2023-11-14T00:57:43.288Z INFO 7 --- [ main] com.zaxxer.hikari.HikariDataSource : HikariPool-2 - Start completed.
2023-11-14T00:57:47.286Z DEBUG 7 --- [ main] o.s.c.t.r.support.SimpleTaskRepository : Starting: TaskExecution{executionId=170, parentExecutionId=null, exitCode=null, taskName='a1-integration-task-batch', startTime=2023-11-14T00:57:47.002589929, endTime=null, exitMessage='null', externalExecutionId='null', errorMessage='null', arguments=[--spring.datasource.driverCl
2023-11-14T00:57:47.486Z INFO 7 --- [ main] o.s.b.w.embedded.tomcat.TomcatWebServer : Tomcat started on port(s): 8080 (http) with context path ''
2023-11-14T00:57:47.496Z INFO 7 --- [ main] c.i.IntegrationApplication$Companion : Started IntegrationApplication.Companion in 46.597 seconds (process running for 66.779)
2023-11-14T00:57:47.502Z INFO 7 --- [ main] BatchServiceImpl : jobRunner : {startDate=2023-11-13}
2023-11-14T00:57:47.502Z INFO 7 --- [ main] BatchServiceImpl : key : startDate
2023-11-14T00:57:47.502Z INFO 7 --- [ main] BatchServiceImpl : jobDataMap[key] : 2023-11-13
2023-11-14T00:57:47.503Z INFO 7 --- [ main] BatchServiceImpl : param.getJobNm() : dailyTradeIntegrationJob
2023-11-14T00:57:47.505Z INFO 7 --- [ main] BatchServiceImpl : jpb.toJobParameters() : {'startDate':'{value=2023-11-13, type=class java.lang.String, identifying=true}'}
2023-11-14T00:57:47.786Z INFO 7 --- [ main] o.s.b.c.l.support.SimpleJobLauncher : Job: [SimpleJob: [name=dailyTradeIntegrationJob]] launched with the following parameters: [{'startDate':'{value=2023-11-13, type=class java.lang.String, identifying=true}'}]
2023-11-14T00:57:47.811Z INFO 7 --- [ main] o.s.c.t.b.l.TaskBatchExecutionListener : The job execution id 3 was run within the task execution 170
2023-11-14T00:57:47.999Z WARN 7 --- [ main] o.s.b.c.l.AbstractListenerFactoryBean : org.springframework.batch.item.ItemReader is an interface. The implementing class will not be queried for annotation based listener configurations. If using @StepScope on a @Bean method, be sure to return the implementing class so listener annotations can be used.
2023-11-14T00:57:48.185Z INFO 7 --- [ main] o.s.batch.core.job.SimpleStepHandler : Executing step: [tradeBidStep]
2023-11-14T00:57:48.204Z INFO 7 --- [ main] TradeHistoryListener : trade history timestamp : 2023-11-14
2023-11-14T00:57:48.205Z INFO 7 --- [ main] com.zaxxer.hikari.HikariDataSource : HikariPool-3 - Starting...
2023-11-14T00:57:48.291Z INFO 7 --- [ main] com.zaxxer.hikari.pool.HikariPool : HikariPool-3 - Added connection com.mysql.cj.jdbc.ConnectionImpl@1ee37c7a
2023-11-14T00:57:48.291Z INFO 7 --- [ main] com.zaxxer.hikari.HikariDataSource : HikariPool-3 - Start completed.
2023-11-14T00:57:48.393Z INFO 7 --- [ main] TradeIntegrationBatchJob : trade history bid startId : 250493683
2023-11-14T00:57:48.393Z INFO 7 --- [ main] TradeIntegrationBatchJob : trade history bid endId : 250605451
2023-11-14T00:57:48.403Z INFO 7 --- [ main] com.zaxxer.hikari.HikariDataSource : HikariPool-4 - Starting...
2023-11-14T00:57:48.503Z INFO 7 --- [ main] com.zaxxer.hikari.pool.HikariPool : HikariPool-4 - Added connection com.mysql.cj.jdbc.ConnectionImpl@7fdaeb6b
2023-11-14T00:57:48.503Z INFO 7 --- [ main] com.zaxxer.hikari.HikariDataSource : HikariPool-4 - Start completed.
2023-11-14T00:57:51.096Z INFO 7 --- [ main] com.zaxxer.hikari.HikariDataSource : HikariPool-5 - Starting...
2023-11-14T00:57:51.196Z INFO 7 --- [ main] com.zaxxer.hikari.pool.HikariPool : HikariPool-5 - Added connection com.mysql.cj.jdbc.ConnectionImpl@4620e669
2023-11-14T00:57:51.196Z INFO 7 --- [ main] com.zaxxer.hikari.HikariDataSource : HikariPool-5 - Start completed.
2023-11-14T00:58:11.067Z INFO 7 --- [ main] o.s.batch.core.step.AbstractStep : Step: [tradeBidStep] executed in 22s880ms
2023-11-14T00:58:11.090Z WARN 7 --- [ main] o.s.b.c.l.AbstractListenerFactoryBean : org.springframework.batch.item.ItemReader is an interface. The implementing class will not be queried for annotation based listener configurations. If using @StepScope on a @Bean method, be sure to return the implementing class so listener annotations can be used.
2023-11-14T00:58:11.119Z INFO 7 --- [ main] o.s.batch.core.job.SimpleStepHandler : Executing step: [tradeAskStep]
2023-11-14T00:58:11.193Z INFO 7 --- [ main] TradeHistoryListener : trade history timestamp : 2023-11-14
2023-11-14T00:58:11.208Z INFO 7 --- [ main] TradeIntegrationBatchJob : trade history ask startId : 250493683
2023-11-14T00:58:11.208Z INFO 7 --- [ main] TradeIntegrationBatchJob : trade history ask endId : 250605451
2023-11-14T00:58:23.133Z INFO 7 --- [ main] o.s.batch.core.step.AbstractStep : Step: [tradeAskStep] executed in 12s13ms
2023-11-14T00:58:23.177Z INFO 7 --- [ main] o.s.batch.core.job.SimpleStepHandler : Executing step: [coinInfoStep]
2023-11-14T00:58:23.213Z INFO 7 --- [ main] o.s.batch.core.step.AbstractStep : Step: [coinInfoStep] executed in 35ms
2023-11-14T00:58:23.228Z WARN 7 --- [ main] o.s.b.c.l.AbstractListenerFactoryBean : org.springframework.batch.item.ItemReader is an interface. The implementing class will not be queried for annotation based listener configurations. If using @StepScope on a @Bean method, be sure to return the implementing class so listener annotations can be used.
2023-11-14T00:57:34.306Z INFO 7 --- [ main] o.a.c.c.C.[Tomcat].[localhost].[/] : Initializing Spring embedded WebApplicationContext
2023-11-14T00:57:34.306Z INFO 7 --- [ main] w.s.c.ServletWebServerApplicationContext : Root WebApplicationContext: initialization completed in 16418 ms
2023-11-14T00:57:38.090Z INFO 7 --- [ main] com.zaxxer.hikari.HikariDataSource : HikariPool-1 - Starting...
2023-11-14T00:57:40.789Z INFO 7 --- [ main] com.zaxxer.hikari.pool.HikariPool : HikariPool-1 - Added connection com.mysql.cj.jdbc.ConnectionImpl@15c8b75c
2023-11-14T00:57:40.792Z INFO 7 --- [ main] com.zaxxer.hikari.HikariDataSource : HikariPool-1 - Start completed.
2023-11-14T00:57:41.094Z DEBUG 7 --- [ main] o.s.c.t.c.SimpleTaskAutoConfiguration : Using xxxxxxx.integration.config.task.TaskConfiguration$$SpringCGLIB$$0 TaskConfigurer
2023-11-14T00:57:41.095Z DEBUG 7 --- [ main] o.s.c.t.c.DefaultTaskConfigurer : No EntityManager was found, using DataSourceTransactionManager
2023-11-14T00:57:43.190Z INFO 7 --- [ main] com.zaxxer.hikari.HikariDataSource : HikariPool-2 - Starting...
2023-11-14T00:57:43.288Z INFO 7 --- [ main] com.zaxxer.hikari.pool.HikariPool : HikariPool-2 - Added connection com.mysql.cj.jdbc.ConnectionImpl@27558dee
2023-11-14T00:57:43.288Z INFO 7 --- [ main] com.zaxxer.hikari.HikariDataSource : HikariPool-2 - Start completed.
2023-11-14T00:57:47.286Z DEBUG 7 --- [ main] o.s.c.t.r.support.SimpleTaskRepository : Starting: TaskExecution{executionId=170, parentExecutionId=null, exitCode=null, taskName='a1-integration-task-batch', startTime=2023-11-14T00:57:47.002589929, endTime=null, exitMessage='null', externalExecutionId='null', errorMessage='null', arguments=[--spring.datasource.driverCl
2023-11-14T00:57:47.486Z INFO 7 --- [ main] o.s.b.w.embedded.tomcat.TomcatWebServer : Tomcat started on port(s): 8080 (http) with context path ''
2023-11-14T00:57:47.496Z INFO 7 --- [ main] c.i.IntegrationApplication$Companion : Started IntegrationApplication.Companion in 46.597 seconds (process running for 66.779)
2023-11-14T00:57:47.502Z INFO 7 --- [ main] BatchServiceImpl : jobRunner : {startDate=2023-11-13}
2023-11-14T00:57:47.502Z INFO 7 --- [ main] BatchServiceImpl : key : startDate
2023-11-14T00:57:47.502Z INFO 7 --- [ main] BatchServiceImpl : jobDataMap[key] : 2023-11-13
2023-11-14T00:57:47.503Z INFO 7 --- [ main] BatchServiceImpl : param.getJobNm() : dailyTradeIntegrationJob
2023-11-14T00:57:47.505Z INFO 7 --- [ main] BatchServiceImpl : jpb.toJobParameters() : {'startDate':'{value=2023-11-13, type=class java.lang.String, identifying=true}'}
2023-11-14T00:57:47.786Z INFO 7 --- [ main] o.s.b.c.l.support.SimpleJobLauncher : Job: [SimpleJob: [name=dailyTradeIntegrationJob]] launched with the following parameters: [{'startDate':'{value=2023-11-13, type=class java.lang.String, identifying=true}'}]
2023-11-14T00:57:47.811Z INFO 7 --- [ main] o.s.c.t.b.l.TaskBatchExecutionListener : The job execution id 3 was run within the task execution 170
2023-11-14T00:57:47.999Z WARN 7 --- [ main] o.s.b.c.l.AbstractListenerFactoryBean : org.springframework.batch.item.ItemReader is an interface. The implementing class will not be queried for annotation based listener configurations. If using @StepScope on a @Bean method, be sure to return the implementing class so listener annotations can be used.
2023-11-14T00:57:48.185Z INFO 7 --- [ main] o.s.batch.core.job.SimpleStepHandler : Executing step: [tradeBidStep]
2023-11-14T00:57:48.204Z INFO 7 --- [ main] TradeHistoryListener : trade history timestamp : 2023-11-14
2023-11-14T00:57:48.205Z INFO 7 --- [ main] com.zaxxer.hikari.HikariDataSource : HikariPool-3 - Starting...
2023-11-14T00:57:48.291Z INFO 7 --- [ main] com.zaxxer.hikari.pool.HikariPool : HikariPool-3 - Added connection com.mysql.cj.jdbc.ConnectionImpl@1ee37c7a
2023-11-14T00:57:48.291Z INFO 7 --- [ main] com.zaxxer.hikari.HikariDataSource : HikariPool-3 - Start completed.
2023-11-14T00:57:48.393Z INFO 7 --- [ main] TradeIntegrationBatchJob : trade history bid startId : 250493683
2023-11-14T00:57:48.393Z INFO 7 --- [ main] TradeIntegrationBatchJob : trade history bid endId : 250605451
2023-11-14T00:57:48.403Z INFO 7 --- [ main] com.zaxxer.hikari.HikariDataSource : HikariPool-4 - Starting...
2023-11-14T00:57:48.503Z INFO 7 --- [ main] com.zaxxer.hikari.pool.HikariPool : HikariPool-4 - Added connection com.mysql.cj.jdbc.ConnectionImpl@7fdaeb6b
2023-11-14T00:57:48.503Z INFO 7 --- [ main] com.zaxxer.hikari.HikariDataSource : HikariPool-4 - Start completed.
2023-11-14T00:57:51.096Z INFO 7 --- [ main] com.zaxxer.hikari.HikariDataSource : HikariPool-5 - Starting...
2023-11-14T00:57:51.196Z INFO 7 --- [ main] com.zaxxer.hikari.pool.HikariPool : HikariPool-5 - Added connection com.mysql.cj.jdbc.ConnectionImpl@4620e669
2023-11-14T00:57:51.196Z INFO 7 --- [ main] com.zaxxer.hikari.HikariDataSource : HikariPool-5 - Start completed.
2023-11-14T00:58:11.067Z INFO 7 --- [ main] o.s.batch.core.step.AbstractStep : Step: [tradeBidStep] executed in 22s880ms
2023-11-14T00:58:11.090Z WARN 7 --- [ main] o.s.b.c.l.AbstractListenerFactoryBean : org.springframework.batch.item.ItemReader is an interface. The implementing class will not be queried for annotation based listener configurations. If using @StepScope on a @Bean method, be sure to return the implementing class so listener annotations can be used.
2023-11-14T00:58:11.119Z INFO 7 --- [ main] o.s.batch.core.job.SimpleStepHandler : Executing step: [tradeAskStep]
2023-11-14T00:58:11.193Z INFO 7 --- [ main] TradeHistoryListener : trade history timestamp : 2023-11-14
2023-11-14T00:58:11.208Z INFO 7 --- [ main] TradeIntegrationBatchJob : trade history ask startId : 250493683
2023-11-14T00:58:11.208Z INFO 7 --- [ main] TradeIntegrationBatchJob : trade history ask endId : 250605451
2023-11-14T00:58:23.133Z INFO 7 --- [ main] o.s.batch.core.step.AbstractStep : Step: [tradeAskStep] executed in 12s13ms
2023-11-14T00:58:23.177Z INFO 7 --- [ main] o.s.batch.core.job.SimpleStepHandler : Executing step: [coinInfoStep]
2023-11-14T00:58:23.213Z INFO 7 --- [ main] o.s.batch.core.step.AbstractStep : Step: [coinInfoStep] executed in 35ms
2023-11-14T00:58:23.228Z WARN 7 --- [ main] o.s.b.c.l.AbstractListenerFactoryBean : org.springframework.batch.item.ItemReader is an interface. The implementing class will not be queried for annotation based listener configurations. If using @StepScope on a @Bean method, be sure to return the implementing class so listener annotations can be used.
2023-11-14T00:58:23.293Z INFO 7 --- [ main] o.s.batch.core.job.SimpleStepHandler : Executing step: [coinTransferHistoryStep]
2023-11-14T00:58:23.303Z INFO 7 --- [ main] CoinTransferListener : coin history fromDateTime : 2023-11-12T15:00
2023-11-14T00:58:23.303Z INFO 7 --- [ main] CoinTransferListener : coin history toDateTime : 2023-11-13T14:59:59
2023-11-14T00:58:24.543Z INFO 7 --- [ main] TradeIntegrationBatchJob : coin history startId : 29945754
2023-11-14T00:58:24.543Z INFO 7 --- [ main] TradeIntegrationBatchJob : coin history endId : 30782305
2023-11-14T00:58:24.543Z INFO 7 --- [ main] TradeIntegrationBatchJob : coin history fromDateTime : 2023-11-12 15:00:00
2023-11-14T00:58:24.543Z INFO 7 --- [ main] TradeIntegrationBatchJob : coin history toDateTime : 2023-11-13 14:59:59
2023-11-14T00:58:40.124Z INFO 7 --- [ main] o.s.batch.core.step.AbstractStep : Step: [coinTransferHistoryStep] executed in 16s831ms
2023-11-14T00:58:40.139Z WARN 7 --- [ main] o.s.b.c.l.AbstractListenerFactoryBean : org.springframework.batch.item.ItemReader is an interface. The implementing class will not be queried for annotation based listener configurations. If using @StepScope on a @Bean method, be sure to return the implementing class so listener annotations can be used.
2023-11-14T00:58:40.169Z INFO 7 --- [ main] o.s.batch.core.job.SimpleStepHandler : Executing step: [krwTransferHistoryStep]
2023-11-14T00:58:40.193Z INFO 7 --- [ main] KrwTransferListener : krw history fromDateTime : 2023-11-12T15:00
2023-11-14T00:58:40.193Z INFO 7 --- [ main] KrwTransferListener : krw history toDateTime : 2023-11-13T14:59:59
2023-11-14T00:58:40.293Z INFO 7 --- [ main] TradeIntegrationBatchJob : krw history startId : 0
2023-11-14T00:58:40.293Z INFO 7 --- [ main] TradeIntegrationBatchJob : krw history endId : 0
2023-11-14T00:58:40.293Z INFO 7 --- [ main] TradeIntegrationBatchJob : krw history fromDateTime : 2023-11-12 15:00:00
2023-11-14T00:58:40.293Z INFO 7 --- [ main] TradeIntegrationBatchJob : krw history toDateTime : 2023-11-13 14:59:59
2023-11-14T00:58:40.313Z INFO 7 --- [ main] o.s.batch.core.step.AbstractStep : Step: [krwTransferHistoryStep] executed in 143ms
2023-11-14T00:58:40.398Z INFO 7 --- [ main] o.s.b.c.l.support.SimpleJobLauncher : Job: [SimpleJob: [name=dailyTradeIntegrationJob]] completed with the following parameters: [{'startDate':'{value=2023-11-13, type=class java.lang.String, identifying=true}'}] and the following status: [COMPLETED] in 52s591ms
2023-11-14T00:58:40.405Z DEBUG 7 --- [ main] o.s.c.t.r.support.SimpleTaskRepository : Updating: TaskExecution with executionId=170 with the following {exitCode=0, endTime=2023-11-14T00:58:40.398760617, exitMessage='null', errorMessage='null'}
2023-11-14T00:58:40.496Z INFO 7 --- [ main] o.apache.catalina.core.StandardService : Stopping service [Tomcat]
2023-11-14T00:58:40.592Z INFO 7 --- [ main] com.zaxxer.hikari.HikariDataSource : HikariPool-1 - Shutdown initiated...
2023-11-14T00:58:40.596Z INFO 7 --- [ main] com.zaxxer.hikari.HikariDataSource : HikariPool-1 - Shutdown completed.
2023-11-14T00:58:40.596Z INFO 7 --- [ main] com.zaxxer.hikari.HikariDataSource : HikariPool-5 - Shutdown initiated...
2023-11-14T00:58:40.598Z INFO 7 --- [ main] com.zaxxer.hikari.HikariDataSource : HikariPool-5 - Shutdown completed.
2023-11-14T00:58:40.598Z INFO 7 --- [ main] com.zaxxer.hikari.HikariDataSource : HikariPool-2 - Shutdown initiated...
2023-11-14T00:58:40.600Z INFO 7 --- [ main] com.zaxxer.hikari.HikariDataSource : HikariPool-2 - Shutdown completed.
2023-11-14T00:58:40.600Z INFO 7 --- [ main] com.zaxxer.hikari.HikariDataSource : HikariPool-4 - Shutdown initiated...
2023-11-14T00:58:40.686Z INFO 7 --- [ main] com.zaxxer.hikari.HikariDataSource : HikariPool-4 - Shutdown completed.
2023-11-14T00:58:40.686Z INFO 7 --- [ main] com.zaxxer.hikari.HikariDataSource : HikariPool-3 - Shutdown initiated...
2023-11-14T00:58:40.688Z INFO 7 --- [ main] com.zaxxer.hikari.HikariDataSource : HikariPool-3 - Shutdown completed.In this task pod logs, the task a1-integration-task-batch_0 was completed.
However if the task is executed after being included in the Composed Task, it will run in the boot2 version and the composed task failed.
cppwfs
commented
1 year ago At the time you register the application integration-task-batch_0 are you selecting Boot3 as its Boot version?
It seems via our discussions that you are setting this manually for each execution and should not have to do this.
younsl
commented
1 year ago Yes.
When creating the applications, I used Spring boot 3.x.
Even after recreating composed-task and applications, I noticed that the same error occurred.
The composed-task a1 failed previously:
a1 composed-task combines integration-task-batch and task-alarm-composed.
The two individual applications that make up composed-task are Spring Boot version 3.x.
Individual task a1-integration-task-batch was completed without problem by boot3 version. Comment
cppwfs
commented
1 year ago Can you provide a sample project for the application integration-task-batch so that we can recreate the scenario on our side?
younsl
commented
1 year ago @cppwfs I'm so sorry. The docker Image used in the SCDF Application contains our enterprise code and base image, so cannot be made public. Could there be something else instead to solve this problem?
corneil
commented
1 year ago Can you provide examples of the registrations of the tasks and text of the composed task definitions?
younsl
commented
1 year ago Some codes are redactedREDACTED as it is important information in the Enterprise level.
[1] integration-task-batch (Application)
This task is executed by private docker container
Type: TASK
Version: latest
Spring Boot version: 3.x
URI: docker:111122223333.dkr.ecr.ap-northeast-2.amazonaws.com/trade-integration:latest
Metadata URI: N/A
Version(s): 1Dockerfile
FROM ecr.example.com/base-images/company-alpine-corretto-jdk17-config-newrelic:0.0.1
COPY ./deploy/trade-integration-batch.jar trade-integration-batch.jar
ENV SPRING_PROPERTIES_ACTIVE dev
ENV SPRING_APPLICATION_NAME TRADE-INTEGRATION-BATCH
ENV SPRING_CONFIG_URL http://<REDACTED>.<REDACTED>
COPY newrelic.yml newrelic.yml
COPY docker-entrypoint.sh docker-entrypoint.sh
EXPOSE 8080
ENTRYPOINT ["sh", "docker-entrypoint.sh"][2] task-alarm-composed (Application)
This task is executed by private docker container
Type: TASK
Version: latest
Spring Boot version: 3.x
URI: docker:111122223333.dkr.ecr.ap-northeast-2.amazonaws.com/task-alarm:latest
Metadata URI: N/A
Version(s): 1FROM ecr.example.com/base-images/company-alpine-corretto-jdk17-config-newrelic:0.0.1
ARG BUILD_DIR=./build/libs
ARG BUILD_JAR=task-alarm.jar
ENV SPRING_PROPERTIES_ACTIVE ${SPRING_PROPERTIES_ACTIVE:-dev}
ENV SPRING_APPLICATION_NAME ${SPRING_APPLICATION_NAME:-BO-DATA-FLOW-TASK}
ENV SPRING_CONFIG_URL ${SPRING_CONFIG_URL:-http://<REDACTED>.<REDACTED>}
COPY newrelic.yml newrelic.yml
COPY docker-entrypoint.sh docker-entrypoint.sh
COPY ${BUILD_DIR}/${BUILD_JAR} app.jar
EXPOSE 8080
ENTRYPOINT ["sh", "docker-entrypoint.sh"]integration-task-batch && task-alarm-composedThe composed task runs the two tasks created above.
younsl
commented
1 year ago Is there any further progress or additional solutions to try?
cppwfs
commented
1 year ago Reviewing the issue again I missed the 401 error you mentioned above. Since it appears you are running SCDF with security enabled, you will need to set additional security properties for CTR when launching it. More can be read about it here: https://dataflow.spring.io/docs/feature-guides/batch/composed-task/#launching-a-composed-task-when-security-is-enabled
younsl
commented
1 year ago @cppwfs I used Data Flow-provided user token as the CTR authentication method to solve this composed-task error.
Referring to the Composed Task Runner document, I added the Argument as shown below and launch the composed-task.
--dataflow-server-use-user-access-token=trueHowever, the following error occurs in Composed Task Runner Pod.
2023-12-05 01:57:59.056 ERROR 1 --- [ main] o.s.batch.core.step.AbstractStep : Encountered an error executing step a1-integration-task-batch_0 in job a1
org.springframework.web.client.HttpClientErrorException$Unauthorized: 401 : [no body]Full logs from composed task runner pod:
scdf-composed-task-runner 01:57:29.07
scdf-composed-task-runner 01:57:29.08 Welcome to the Bitnami spring-cloud-dataflow-composed-task-runner container
scdf-composed-task-runner 01:57:29.08 Subscribe to project updates by watching https://github.com/bitnami/containers
scdf-composed-task-runner 01:57:29.08 Submit issues and feature requests at https://github.com/bitnami/containers/issues
scdf-composed-task-runner 01:57:29.08
scdf-composed-task-runner 01:57:29.09 INFO ==> ** Starting Spring Cloud Dataflow Compose Task Runner **
_____ _ _______ _
/ ____| | | |__ __| | |
| | ___ _ __ ___ _ __ ___ ___ ___ __| | | | __ _ ___| | __
| | / _ \| '_ ` _ \| '_ \ / _ \/ __|/ _ \/ _` | | |/ _` / __| |/ /
| |___| (_) | | | | | | |_) | (_) \__ \ __/ (_| | | | (_| \__ \ <
\_____\___/|_| |_| |_| .__/ \___/|___/\___|\__,_| |_|\__,_|___/_|\_\
| |
|_|
_____
| __ \
| |__) | _ _ __ _ __ ___ _ __
| _ / | | | '_ \| '_ \ / _ \ '__|
| | \ \ |_| | | | | | | | __/ |
|_| \_\__,_|_| |_|_| |_|\___|_|
2023-12-05 01:57:33.752 INFO 1 --- [ main] o.s.c.d.c.ComposedTaskRunner : Starting ComposedTaskRunner using Java 17.0.8 on a1-8yqemzdmd4 with PID 1 (/opt/bitnami/spring-cloud-dataflow-composed-task-runner/spring-cloud-dataflow-composed-task-runner-2.11.1.jar started by ? in /)
2023-12-05 01:57:33.755 INFO 1 --- [ main] o.s.c.d.c.ComposedTaskRunner : No active profile set, falling back to 1 default profile: "default"
2023-12-05 01:57:38.153 INFO 1 --- [ main] .s.d.r.c.RepositoryConfigurationDelegate : Multiple Spring Data modules found, entering strict repository configuration mode
2023-12-05 01:57:38.155 INFO 1 --- [ main] .s.d.r.c.RepositoryConfigurationDelegate : Bootstrapping Spring Data JPA repositories in DEFAULT mode.
2023-12-05 01:57:38.256 INFO 1 --- [ main] .s.d.r.c.RepositoryConfigurationDelegate : Finished Spring Data repository scanning in 93 ms. Found 0 JPA repository interfaces.
2023-12-05 01:57:40.052 INFO 1 --- [ main] trationDelegate$BeanPostProcessorChecker : Bean 'org.springframework.cloud.task.batch.configuration.TaskBatchAutoConfiguration' of type [org.springframework.cloud.task.batch.configuration.TaskBatchAutoConfiguration] is not eligible for getting processed by all BeanPostProcessors (for example: not eligible for auto-proxying)
2023-12-05 01:57:41.556 INFO 1 --- [ main] com.zaxxer.hikari.HikariDataSource : HikariPool-1 - Starting...
2023-12-05 01:57:42.255 INFO 1 --- [ main] com.zaxxer.hikari.HikariDataSource : HikariPool-1 - Start completed.
2023-12-05 01:57:42.951 INFO 1 --- [ main] o.hibernate.jpa.internal.util.LogHelper : HHH000204: Processing PersistenceUnitInfo [name: default]
2023-12-05 01:57:43.254 INFO 1 --- [ main] org.hibernate.Version : HHH000412: Hibernate ORM core version 5.6.15.Final
2023-12-05 01:57:43.972 INFO 1 --- [ main] o.hibernate.annotations.common.Version : HCANN000001: Hibernate Commons Annotations {5.1.2.Final}
2023-12-05 01:57:44.665 INFO 1 --- [ main] org.hibernate.dialect.Dialect : HHH000400: Using dialect: org.hibernate.dialect.MariaDB53Dialect
2023-12-05 01:57:46.062 INFO 1 --- [ main] o.h.e.t.j.p.i.JtaPlatformInitiator : HHH000490: Using JtaPlatform implementation: [org.hibernate.engine.transaction.jta.platform.internal.NoJtaPlatform]
2023-12-05 01:57:46.149 INFO 1 --- [ main] j.LocalContainerEntityManagerFactoryBean : Initialized JPA EntityManagerFactory for persistence unit 'default'
2023-12-05 01:57:46.786 INFO 1 --- [ main] .c.d.c.ComposedTaskStepExecutionListener : ComposedTaskStepExecutionListener supporting [a1-integration-task-batch, integration-task-batch, a1-task-alarm-composed, task-alarm-composed]
2023-12-05 01:57:46.870 INFO 1 --- [ main] o.s.b.c.r.s.JobRepositoryFactoryBean : No database type set, using meta data indicating: MYSQL
2023-12-05 01:57:46.981 INFO 1 --- [ main] o.s.b.c.r.s.JobRepositoryFactoryBean : No database type set, using meta data indicating: MYSQL
2023-12-05 01:57:46.997 INFO 1 --- [ main] o.s.b.c.l.support.SimpleJobLauncher : No TaskExecutor has been set, defaulting to synchronous executor.
2023-12-05 01:57:48.759 INFO 1 --- [ main] o.s.b.a.e.web.EndpointLinksResolver : Exposing 1 endpoint(s) beneath base path '/actuator'
2023-12-05 01:57:55.653 WARN 1 --- [ main] i.m.p.rsocket.PrometheusRSocketClient : Creating the connection and receiving the key timed out!
2023-12-05 01:57:56.063 INFO 1 --- [ main] ctiveUserDetailsServiceAutoConfiguration :
Using generated security password: XXX-XXX-XXX-XXX-XXX
2023-12-05 01:57:56.954 INFO 1 --- [ main] o.s.c.d.c.ComposedTaskRunnerTaskListener : onTaskStartup:executionId=137
2023-12-05 01:57:56.965 INFO 1 --- [ main] o.s.b.web.embedded.netty.NettyWebServer : Netty started on port 8080
2023-12-05 01:57:57.060 INFO 1 --- [ main] o.s.c.d.c.ComposedTaskRunner : Started ComposedTaskRunner in 25.991 seconds (JVM running for 27.959)
2023-12-05 01:57:57.063 INFO 1 --- [ main] o.s.b.a.b.JobLauncherApplicationRunner : Running default command line with: []
2023-12-05 01:57:57.064 INFO 1 --- [ main] o.s.b.c.r.s.JobRepositoryFactoryBean : No database type set, using meta data indicating: MYSQL
2023-12-05 01:57:57.454 INFO 1 --- [ main] o.s.b.c.l.support.SimpleJobLauncher : Job: [FlowJob: [name=a1]] launched with the following parameters: [{run.id=4}]
2023-12-05 01:57:57.464 INFO 1 --- [ main] o.s.c.t.b.l.TaskBatchExecutionListener : The job execution id 64 was run within the task execution 137
2023-12-05 01:57:57.552 INFO 1 --- [ main] o.s.batch.core.job.SimpleStepHandler : Executing step: [a1-integration-task-batch_0]
2023-12-05 01:57:57.559 INFO 1 --- [ main] .c.d.c.ComposedTaskStepExecutionListener : beforeStep:a1-integration-task-batch_0:64>>>>
2023-12-05 01:57:57.560 INFO 1 --- [ main] .c.d.c.ComposedTaskStepExecutionListener : beforeStep:a1-integration-task-batch_0:64<<<
2023-12-05 01:57:59.056 ERROR 1 --- [ main] o.s.batch.core.step.AbstractStep : Encountered an error executing step a1-integration-task-batch_0 in job a1
org.springframework.web.client.HttpClientErrorException$Unauthorized: 401 : [no body]
at org.springframework.web.client.HttpClientErrorException.create(HttpClientErrorException.java:105) ~[spring-web-5.3.30.jar!/:5.3.30]
at org.springframework.web.client.DefaultResponseErrorHandler.handleError(DefaultResponseErrorHandler.java:168) ~[spring-web-5.3.30.jar!/:5.3.30]
at org.springframework.web.client.DefaultResponseErrorHandler.handleError(DefaultResponseErrorHandler.java:122) ~[spring-web-5.3.30.jar!/:5.3.30]
at org.springframework.cloud.dataflow.rest.client.VndErrorResponseErrorHandler.handleError(VndErrorResponseErrorHandler.java:62) ~[spring-cloud-dataflow-rest-client-2.11.1.jar!/:na]
at org.springframework.web.client.ResponseErrorHandler.handleError(ResponseErrorHandler.java:63) ~[spring-web-5.3.30.jar!/:5.3.30]
at org.springframework.web.client.RestTemplate.handleResponse(RestTemplate.java:825) ~[spring-web-5.3.30.jar!/:5.3.30]
at org.springframework.web.client.RestTemplate.doExecute(RestTemplate.java:783) ~[spring-web-5.3.30.jar!/:5.3.30]
at org.springframework.web.client.RestTemplate.execute(RestTemplate.java:757) ~[spring-web-5.3.30.jar!/:5.3.30]
at org.springframework.web.client.RestTemplate.getForObject(RestTemplate.java:358) ~[spring-web-5.3.30.jar!/:5.3.30]
at org.springframework.cloud.dataflow.rest.client.DataFlowTemplate.<init>(DataFlowTemplate.java:150) ~[spring-cloud-dataflow-rest-client-2.11.1.jar!/:na]
at org.springframework.cloud.dataflow.composedtaskrunner.TaskLauncherTasklet.dataFlowOperations(TaskLauncherTasklet.java:327) ~[classes!/:na]
at org.springframework.cloud.dataflow.composedtaskrunner.TaskLauncherTasklet.taskOperations(TaskLauncherTasklet.java:265) ~[classes!/:na]
at org.springframework.cloud.dataflow.composedtaskrunner.TaskLauncherTasklet.execute(TaskLauncherTasklet.java:163) ~[classes!/:na]
at org.springframework.batch.core.step.tasklet.TaskletStep$ChunkTransactionCallback.doInTransaction(TaskletStep.java:407) ~[spring-batch-core-4.3.9.jar!/:4.3.9]
at org.springframework.batch.core.step.tasklet.TaskletStep$ChunkTransactionCallback.doInTransaction(TaskletStep.java:331) ~[spring-batch-core-4.3.9.jar!/:4.3.9]
at org.springframework.transaction.support.TransactionTemplate.execute(TransactionTemplate.java:140) ~[spring-tx-5.3.30.jar!/:5.3.30]
at org.springframework.batch.core.step.tasklet.TaskletStep$2.doInChunkContext(TaskletStep.java:273) ~[spring-batch-core-4.3.9.jar!/:4.3.9]
at org.springframework.batch.core.scope.context.StepContextRepeatCallback.doInIteration(StepContextRepeatCallback.java:82) ~[spring-batch-core-4.3.9.jar!/:4.3.9]
at org.springframework.batch.repeat.support.RepeatTemplate.getNextResult(RepeatTemplate.java:375) ~[spring-batch-infrastructure-4.3.9.jar!/:4.3.9]
at org.springframework.batch.repeat.support.RepeatTemplate.executeInternal(RepeatTemplate.java:215) ~[spring-batch-infrastructure-4.3.9.jar!/:4.3.9]
at org.springframework.batch.repeat.support.RepeatTemplate.iterate(RepeatTemplate.java:145) ~[spring-batch-infrastructure-4.3.9.jar!/:4.3.9]
at org.springframework.batch.core.step.tasklet.TaskletStep.doExecute(TaskletStep.java:258) ~[spring-batch-core-4.3.9.jar!/:4.3.9]
at org.springframework.batch.core.step.AbstractStep.execute(AbstractStep.java:208) ~[spring-batch-core-4.3.9.jar!/:4.3.9]
at org.springframework.batch.core.job.SimpleStepHandler.handleStep(SimpleStepHandler.java:152) ~[spring-batch-core-4.3.9.jar!/:4.3.9]
at org.springframework.batch.core.job.flow.JobFlowExecutor.executeStep(JobFlowExecutor.java:68) ~[spring-batch-core-4.3.9.jar!/:4.3.9]
at org.springframework.batch.core.job.flow.support.state.StepState.handle(StepState.java:68) ~[spring-batch-core-4.3.9.jar!/:4.3.9]
at org.springframework.batch.core.job.flow.support.SimpleFlow.resume(SimpleFlow.java:167) ~[spring-batch-core-4.3.9.jar!/:4.3.9]
at org.springframework.batch.core.job.flow.support.SimpleFlow.start(SimpleFlow.java:142) ~[spring-batch-core-4.3.9.jar!/:4.3.9]
at org.springframework.batch.core.job.flow.support.state.FlowState.handle(FlowState.java:56) ~[spring-batch-core-4.3.9.jar!/:4.3.9]
at org.springframework.batch.core.job.flow.support.SimpleFlow.resume(SimpleFlow.java:167) ~[spring-batch-core-4.3.9.jar!/:4.3.9]
at org.springframework.batch.core.job.flow.support.SimpleFlow.start(SimpleFlow.java:142) ~[spring-batch-core-4.3.9.jar!/:4.3.9]
at org.springframework.batch.core.job.flow.FlowJob.doExecute(FlowJob.java:139) ~[spring-batch-core-4.3.9.jar!/:4.3.9]
at org.springframework.batch.core.job.AbstractJob.execute(AbstractJob.java:320) ~[spring-batch-core-4.3.9.jar!/:4.3.9]
at org.springframework.batch.core.launch.support.SimpleJobLauncher$1.run(SimpleJobLauncher.java:149) ~[spring-batch-core-4.3.9.jar!/:4.3.9]
at org.springframework.core.task.SyncTaskExecutor.execute(SyncTaskExecutor.java:50) ~[spring-core-5.3.30.jar!/:5.3.30]
at org.springframework.batch.core.launch.support.SimpleJobLauncher.run(SimpleJobLauncher.java:140) ~[spring-batch-core-4.3.9.jar!/:4.3.9]
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:na]
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77) ~[na:na]
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:na]
at java.base/java.lang.reflect.Method.invoke(Method.java:568) ~[na:na]
at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:344) ~[spring-aop-5.3.30.jar!/:5.3.30]
at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:198) ~[spring-aop-5.3.30.jar!/:5.3.30]
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163) ~[spring-aop-5.3.30.jar!/:5.3.30]
at org.springframework.batch.core.configuration.annotation.SimpleBatchConfiguration$PassthruAdvice.invoke(SimpleBatchConfiguration.java:128) ~[spring-batch-core-4.3.9.jar!/:4.3.9]
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) ~[spring-aop-5.3.30.jar!/:5.3.30]
at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:241) ~[spring-aop-5.3.30.jar!/:5.3.30]
at jdk.proxy2/jdk.proxy2.$Proxy92.run(Unknown Source) ~[na:na]
at org.springframework.boot.autoconfigure.batch.JobLauncherApplicationRunner.execute(JobLauncherApplicationRunner.java:213) ~[spring-boot-autoconfigure-2.7.16.jar!/:2.7.16]
at org.springframework.boot.autoconfigure.batch.JobLauncherApplicationRunner.executeLocalJobs(JobLauncherApplicationRunner.java:193) ~[spring-boot-autoconfigure-2.7.16.jar!/:2.7.16]
at org.springframework.boot.autoconfigure.batch.JobLauncherApplicationRunner.launchJobFromProperties(JobLauncherApplicationRunner.java:172) ~[spring-boot-autoconfigure-2.7.16.jar!/:2.7.16]
at org.springframework.boot.autoconfigure.batch.JobLauncherApplicationRunner.run(JobLauncherApplicationRunner.java:167) ~[spring-boot-autoconfigure-2.7.16.jar!/:2.7.16]
at org.springframework.boot.autoconfigure.batch.JobLauncherApplicationRunner.run(JobLauncherApplicationRunner.java:162) ~[spring-boot-autoconfigure-2.7.16.jar!/:2.7.16]
at org.springframework.boot.SpringApplication.callRunner(SpringApplication.java:759) ~[spring-boot-2.7.16.jar!/:2.7.16]
at org.springframework.boot.SpringApplication.callRunners(SpringApplication.java:749) ~[spring-boot-2.7.16.jar!/:2.7.16]
at org.springframework.boot.SpringApplication.run(SpringApplication.java:314) ~[spring-boot-2.7.16.jar!/:2.7.16]
at org.springframework.boot.SpringApplication.run(SpringApplication.java:1303) ~[spring-boot-2.7.16.jar!/:2.7.16]
at org.springframework.boot.SpringApplication.run(SpringApplication.java:1292) ~[spring-boot-2.7.16.jar!/:2.7.16]
at org.springframework.cloud.dataflow.composedtaskrunner.ComposedTaskRunner.main(ComposedTaskRunner.java:31) ~[classes!/:na]
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:na]
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77) ~[na:na]
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:na]
at java.base/java.lang.reflect.Method.invoke(Method.java:568) ~[na:na]
at org.springframework.boot.loader.MainMethodRunner.run(MainMethodRunner.java:49) ~[spring-cloud-dataflow-composed-task-runner-2.11.1.jar:na]
at org.springframework.boot.loader.Launcher.launch(Launcher.java:108) ~[spring-cloud-dataflow-composed-task-runner-2.11.1.jar:na]
at org.springframework.boot.loader.Launcher.launch(Launcher.java:58) ~[spring-cloud-dataflow-composed-task-runner-2.11.1.jar:na]
at org.springframework.boot.loader.JarLauncher.main(JarLauncher.java:65) ~[spring-cloud-dataflow-composed-task-runner-2.11.1.jar:na]
2023-12-05 01:57:59.057 INFO 1 --- [ main] .c.d.c.ComposedTaskStepExecutionListener : AfterStep processing for stepExecution a1-integration-task-batch_0:64
2023-12-05 01:57:59.057 ERROR 1 --- [ main] o.s.batch.core.step.AbstractStep : Exception in afterStep callback in step a1-integration-task-batch_0 in job a1
java.lang.IllegalArgumentException: TaskLauncherTasklet for job 64 did not return a task-execution-id. Check to see if task exists.
at org.springframework.util.Assert.notNull(Assert.java:201) ~[spring-core-5.3.30.jar!/:5.3.30]
at org.springframework.cloud.dataflow.composedtaskrunner.ComposedTaskStepExecutionListener.afterStep(ComposedTaskStepExecutionListener.java:67) ~[classes!/:na]
at org.springframework.batch.core.listener.CompositeStepExecutionListener.afterStep(CompositeStepExecutionListener.java:64) ~[spring-batch-core-4.3.9.jar!/:4.3.9]
at org.springframework.batch.core.step.AbstractStep.execute(AbstractStep.java:247) ~[spring-batch-core-4.3.9.jar!/:4.3.9]
at org.springframework.batch.core.job.SimpleStepHandler.handleStep(SimpleStepHandler.java:152) ~[spring-batch-core-4.3.9.jar!/:4.3.9]
at org.springframework.batch.core.job.flow.JobFlowExecutor.executeStep(JobFlowExecutor.java:68) ~[spring-batch-core-4.3.9.jar!/:4.3.9]
at org.springframework.batch.core.job.flow.support.state.StepState.handle(StepState.java:68) ~[spring-batch-core-4.3.9.jar!/:4.3.9]
at org.springframework.batch.core.job.flow.support.SimpleFlow.resume(SimpleFlow.java:167) ~[spring-batch-core-4.3.9.jar!/:4.3.9]
at org.springframework.batch.core.job.flow.support.SimpleFlow.start(SimpleFlow.java:142) ~[spring-batch-core-4.3.9.jar!/:4.3.9]
at org.springframework.batch.core.job.flow.support.state.FlowState.handle(FlowState.java:56) ~[spring-batch-core-4.3.9.jar!/:4.3.9]
at org.springframework.batch.core.job.flow.support.SimpleFlow.resume(SimpleFlow.java:167) ~[spring-batch-core-4.3.9.jar!/:4.3.9]
at org.springframework.batch.core.job.flow.support.SimpleFlow.start(SimpleFlow.java:142) ~[spring-batch-core-4.3.9.jar!/:4.3.9]
at org.springframework.batch.core.job.flow.FlowJob.doExecute(FlowJob.java:139) ~[spring-batch-core-4.3.9.jar!/:4.3.9]
at org.springframework.batch.core.job.AbstractJob.execute(AbstractJob.java:320) ~[spring-batch-core-4.3.9.jar!/:4.3.9]
at org.springframework.batch.core.launch.support.SimpleJobLauncher$1.run(SimpleJobLauncher.java:149) ~[spring-batch-core-4.3.9.jar!/:4.3.9]
at org.springframework.core.task.SyncTaskExecutor.execute(SyncTaskExecutor.java:50) ~[spring-core-5.3.30.jar!/:5.3.30]
at org.springframework.batch.core.launch.support.SimpleJobLauncher.run(SimpleJobLauncher.java:140) ~[spring-batch-core-4.3.9.jar!/:4.3.9]
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:na]
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77) ~[na:na]
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:na]
at java.base/java.lang.reflect.Method.invoke(Method.java:568) ~[na:na]
at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:344) ~[spring-aop-5.3.30.jar!/:5.3.30]
at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:198) ~[spring-aop-5.3.30.jar!/:5.3.30]
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163) ~[spring-aop-5.3.30.jar!/:5.3.30]
at org.springframework.batch.core.configuration.annotation.SimpleBatchConfiguration$PassthruAdvice.invoke(SimpleBatchConfiguration.java:128) ~[spring-batch-core-4.3.9.jar!/:4.3.9]
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) ~[spring-aop-5.3.30.jar!/:5.3.30]
at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:241) ~[spring-aop-5.3.30.jar!/:5.3.30]
at jdk.proxy2/jdk.proxy2.$Proxy92.run(Unknown Source) ~[na:na]
at org.springframework.boot.autoconfigure.batch.JobLauncherApplicationRunner.execute(JobLauncherApplicationRunner.java:213) ~[spring-boot-autoconfigure-2.7.16.jar!/:2.7.16]
at org.springframework.boot.autoconfigure.batch.JobLauncherApplicationRunner.executeLocalJobs(JobLauncherApplicationRunner.java:193) ~[spring-boot-autoconfigure-2.7.16.jar!/:2.7.16]
at org.springframework.boot.autoconfigure.batch.JobLauncherApplicationRunner.launchJobFromProperties(JobLauncherApplicationRunner.java:172) ~[spring-boot-autoconfigure-2.7.16.jar!/:2.7.16]
at org.springframework.boot.autoconfigure.batch.JobLauncherApplicationRunner.run(JobLauncherApplicationRunner.java:167) ~[spring-boot-autoconfigure-2.7.16.jar!/:2.7.16]
at org.springframework.boot.autoconfigure.batch.JobLauncherApplicationRunner.run(JobLauncherApplicationRunner.java:162) ~[spring-boot-autoconfigure-2.7.16.jar!/:2.7.16]
at org.springframework.boot.SpringApplication.callRunner(SpringApplication.java:759) ~[spring-boot-2.7.16.jar!/:2.7.16]
at org.springframework.boot.SpringApplication.callRunners(SpringApplication.java:749) ~[spring-boot-2.7.16.jar!/:2.7.16]
at org.springframework.boot.SpringApplication.run(SpringApplication.java:314) ~[spring-boot-2.7.16.jar!/:2.7.16]
at org.springframework.boot.SpringApplication.run(SpringApplication.java:1303) ~[spring-boot-2.7.16.jar!/:2.7.16]
at org.springframework.boot.SpringApplication.run(SpringApplication.java:1292) ~[spring-boot-2.7.16.jar!/:2.7.16]
at org.springframework.cloud.dataflow.composedtaskrunner.ComposedTaskRunner.main(ComposedTaskRunner.java:31) ~[classes!/:na]
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:na]
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77) ~[na:na]
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:na]
at java.base/java.lang.reflect.Method.invoke(Method.java:568) ~[na:na]
at org.springframework.boot.loader.MainMethodRunner.run(MainMethodRunner.java:49) ~[spring-cloud-dataflow-composed-task-runner-2.11.1.jar:na]
at org.springframework.boot.loader.Launcher.launch(Launcher.java:108) ~[spring-cloud-dataflow-composed-task-runner-2.11.1.jar:na]
at org.springframework.boot.loader.Launcher.launch(Launcher.java:58) ~[spring-cloud-dataflow-composed-task-runner-2.11.1.jar:na]
at org.springframework.boot.loader.JarLauncher.main(JarLauncher.java:65) ~[spring-cloud-dataflow-composed-task-runner-2.11.1.jar:na]
2023-12-05 01:57:59.060 INFO 1 --- [ main] o.s.batch.core.step.AbstractStep : Step: [a1-integration-task-batch_0] executed in 1s508ms
2023-12-05 01:57:59.077 INFO 1 --- [ main] o.s.b.c.l.support.SimpleJobLauncher : Job: [FlowJob: [name=a1]] completed with the following parameters: [{run.id=4}] and the following status: [FAILED] in 1s614ms
2023-12-05 01:57:59.161 INFO 1 --- [ main] j.LocalContainerEntityManagerFactoryBean : Closing JPA EntityManagerFactory for persistence unit 'default'
2023-12-05 01:57:59.163 INFO 1 --- [ main] com.zaxxer.hikari.HikariDataSource : HikariPool-1 - Shutdown initiated...
2023-12-05 01:57:59.167 INFO 1 --- [ main] com.zaxxer.hikari.HikariDataSource : HikariPool-1 - Shutdown completed.Data Flow-provided user token as the CTR authentication method is not working for me. Is there something else I've missed?
younsl
commented
11 months ago Is there any further progress or additional solutions to try? Individual SCDF tasks run normally, but I cannot understand why an error occurs only in CTR(Composed Task Runner).
# values.yaml for spring-cloud-dataflow
...
composedTaskRunner:
## Bitnami Spring Cloud Dataflow Composed Task Runner image
## ref: https://hub.docker.com/r/bitnami/spring-cloud-dataflow/tags/
## @param server.composedTaskRunner.image.registry [default: REGISTRY_NAME] Spring Cloud Dataflow Composed Task Runner image registry
## @param server.composedTaskRunner.image.repository [default: REPOSITORY_NAME/spring-cloud-dataflow-composed-task-runner] Spring Cloud Dataflow Composed Task Runner image repository
## @skip server.composedTaskRunner.image.tag Spring Cloud Dataflow Composed Task Runner image tag (immutable tags are recommended)
## @param server.composedTaskRunner.image.digest Spring Cloud Dataflow Composed Task Runner image digest in the way sha256:aa.... Please note this parameter, if set, will override the tag
##
image:
registry: docker.io
repository: bitnami/spring-cloud-dataflow-composed-task-runner
tag: 2.11.1-debian-11-r0
...I recently discovered a workaround.
I think the cause of this problem is not a version issue with the Spring Boot 2.x to 3.x configuration, It appears to be an OAuth authentication issue between the composed task runner and SCDF Server.
I have not yet found a straight solution.
When I ran the CTR(Composed Task Runner), org.springframework.web.client.HttpClientErrorException$Unauthorized: 401 : [no body] error occured and failed running composed task. However, in the case of running individual tasks, run completed boot3.x without any problems.
Comment out the OAuth2 settings to disable them.
Sensitive credentials that should not be exposed are marked as
<REDACTED>.
# spring-cloud-dataflow/server/configmap.yaml
# HELM TEMPLATE
...
data:
application.yaml: |-
{{- if .Values.metrics.enabled }}
{{- $fullname := include "scdf.fullname" . }}
{{- $rsocketPort := int (coalesce .Values.metrics.service.ports.rsocket .Values.metrics.service.rsocketPort) }}
management:
metrics:
export:
prometheus:
enabled: true
rsocket:
enabled: true
host: {{ $fullname }}-prometheus-proxy
port: {{ $rsocketPort }}
{{- end }}
spring:
{{- /* Comment to get workaround 401 authentication issue
security:
basic:
enabled: true
realm: Spring Cloud Data Flow
oauth2:
client:
authorization:
check-token-access: isAuthenticated()
registration:
google:
client-id: <REDACTED>-<REDACTED>.apps.googleusercontent.com
client-secret: XXXXXX-<REDACTED>-<REDACTED>
scope:
- profile
- email
redirect-uri: https://dataflow.example.com/login/oauth2/code/google
authorization-grant-type: authorization_code
provider:
google:
user-info-uri: https://www.googleapis.com/oauth2/v3/userinfo
user-name-attribute: email
*/}}
cloud:
config:
enabled: true
...After redeploying the SCDF Server Pod to apply new configmap, I ran CTR and it ran successfully.
Even if I apply the Data Flow-provided user token method mentioned in the official doc, the same Unauthorized: 401 error occurs in composed task runner pod.
cppwfs
commented
11 months ago Looks like this issue is resolved. If closed by mistake please add a comment. Thank you!
Description:
I run composed-task, but failed with following error messages. Error messages in composed-task:
Application properties in k8s ConfigMap:
I set up Application Properties by referring to the two articles (#5470 #5482).
To change the boot version related arguments to boot3 in composed-task, I try to add the three parameters below referring this comment
spring.cloud.task.schemaTarget = boot3spring.cloud.task.tablePrefix = BOOT3_TASK_spring.cloud.deployer.bootVersion = 3Release versions:
Screenshots:
Screenshot moved after pressing the Launch Task button:
Failed composed-task parameter screenshot:
List of task execution:
Schema Target value is
boot2, instead ofboot3in the Task executions menu.Why is it output as boot2 even though Application Properties is set through ConfigMap? Is there a way to set it up or documents?