spring-cloud / spring-cloud-dataflow

A microservices-based Streaming and Batch data processing in Cloud Foundry and Kubernetes
https://dataflow.spring.io
Apache License 2.0
1.11k stars 580 forks source link

Deployment properties are stored from previous executions but will not be used for next execution #4467

Open canerkoseren opened 3 years ago

canerkoseren commented 3 years ago

Hello,

I have question for deployment properties carried from previous execution.

I executed my task several times with different deployment properties.

In the first execution, I mounted a persistent volume claim by using the properties below to read file from a volume.

deployer.appl.kubernetes.volumeMounts=[{name: 'app-pvc', mountPath: '/data/pvc'}]
deployer.appl.kubernetes.volumes=[{name: 'app-pvc', persistentVolumeClaim: { claimName: 'app-nfs-pvc', readOnly : 'false'}}]

But in the next execution, I did not mount a persistent volume claim, because I would use a jdbc reader, not use a flat file reader. Although I did not set these deployment properties, they were carried from the previous execution. So, the same persistent volume had been mounted to the pod.

To handle this case, I have to register the task with a different task name.

Is it possible to execute the same task with different deployment properties? Are you planning an implementation for this case?

cppwfs commented 3 years ago

Not at this time. But I will put this as a feature on the backlog.

taxone commented 3 years ago

You can set the deployment properties with empty values, so the volumes are not mounted:

deployer.appl.kubernetes.volumeMounts=
deployer.appl.kubernetes.volumes=