The goal of this PR is to remove the dependency on PV from Kubernetes mode.
The approach
Currently PV is used to share the files between Runner and Workflow pods. Those files include scripts that should be executed on runners and required software such as Node.
My assumption is that it's a one way communication from Runner to Workflow, meaning that it's a Runner pod that "sends" file to the "Workflow" and Workflow pod does not need to share any files with Runner.
flowchart LR
A[Runner Pod] -->|Sends files to| B[Workflow Pod]
A -. Pull logs from.-> B
Based on that assumption we can copy the required files directly to Workflow pod without provisioning common storage. We can use cpToPod function for direct file copying.
This PR is a POC that was tested on a workflows that run in a container and it worked fine. I wonder, if there is any flaws that will make this approach unviable.
Implementation Notes
I had to rework cpToPod function to wait for the operation to finish.
The goal of this PR is to remove the dependency on PV from Kubernetes mode.
The approach
Currently PV is used to share the files between Runner and Workflow pods. Those files include scripts that should be executed on runners and required software such as Node.
My assumption is that it's a one way communication from Runner to Workflow, meaning that it's a Runner pod that "sends" file to the "Workflow" and Workflow pod does not need to share any files with Runner.
Based on that assumption we can copy the required files directly to Workflow pod without provisioning common storage. We can use cpToPod function for direct file copying.
This will address the issue described here.
Feedback Request
This PR is a POC that was tested on a workflows that run in a container and it worked fine. I wonder, if there is any flaws that will make this approach unviable.
Implementation Notes
I had to rework
cpToPodfunction to wait for the operation to finish.