argoproj / argo-workflows

Workflow Engine for Kubernetes
https://argo-workflows.readthedocs.io/
Apache License 2.0
15.08k stars 3.2k forks source link

Problem on retryStrategy workflow when it used git artifact #9257

Open DiegoFlop opened 2 years ago

DiegoFlop commented 2 years ago

Checklist

* [x] Double-checked my configuration. * [x] Tested using the latest version. * [x] Used the Emissary executor. ## Summary What happened/what you expected to happen? I'm using field **retryStrategy=3** at workflow template configuration When the workflow fail on a step, this, retry the same step 3 times, if none of them finishes correctly, the workflow retry all steps since start. This is fine, however, on the first step I use the git artifact but when retry the workflow since start. The second time workflow fail because: the files exists from first step, obviously. > Error (exit code 1): rename /mainctrfs/workspace/project.tmp /mainctrfs/workspace/project: file exists ![Captura de pantalla 2022-08-01 a las 9 34 09](https://user-images.githubusercontent.com/34077334/182097778-4e8fc88c-fbc9-4a2e-9243-4c0bf19a885f.png) I would like that You add additional field for example "delete" to remove pre existing content What version are you running? Argo workflows: 3.3.8 Kubernetes: 1.22.8 ## Diagnostics Paste the smallest workflow that reproduces the bug. We must be able to run the workflow. ```yaml apiVersion: argoproj.io/v1alpha1 kind: ClusterWorkflowTemplate metadata: name: step1 spec: serviceAccountName: operate-workflow-sa volumeClaimTemplates: - metadata: name: workdir spec: accessModes: [ "ReadWriteOnce" ] resources: requests: storage: 5Gi entrypoint: main templates: - name: main dag: tasks: - name: checkout templateRef: name: checkout template: main arguments: parameters: - name: git_ssh_url value: "git@{{= sprig.replace('net/', 'net:', sprig.substr( 8, -1, jsonpath(workflow.parameters.EVENT_DATA, '$.project.git_http_url'))) }}" - name: BRANCH value: "{{= jsonpath(workflow.parameters.EVENT_DATA, '$.object_attributes.source_branch')}}" ``` ```yaml apiVersion: argoproj.io/v1alpha1 kind: WorkflowTemplate metadata: name: checkout spec: entrypoint: main templates: - name: main activeDeadlineSeconds: "120" volumes: - name: config-volume configMap: name: argo-workflows inputs: parameters: - name: git_ssh_url - name: BRANCH artifacts: - name: git-clone path: "/workspace/project" git: revision: "{{inputs.parameters.BRANCH}}" repo: "{{inputs.parameters.git_ssh_url}}" singleBranch: false sshPrivateKeySecret: name: github key: id_rsa insecureIgnoreHostKey: true optional: true container: imagePullPolicy: Always volumeMounts: - name: workdir mountPath: "/workspace" - name: config-volume mountPath: /utils image: XXXXXXX:1.0.0 workingDir: "/workspace/project" command: - "python" args: - "xxxxx" - "xxxxx" ``` ```bash # Logs from the workflow controller: kubectl logs -n argo deploy/workflow-controller | grep ${workflow} time="2022-07-29T12:45:37.398Z" level=info msg="Processing workflow" namespace=argo-events workflow=workflow-problem-hm68w time="2022-07-29T12:45:37.437Z" level=info msg="Updated phase -> Running" namespace=argo-events workflow=workflow-problem-hm68w time="2022-07-29T12:45:37.438Z" level=info msg="Creating pvc workflow-problem-hm68w-workdir" namespace=argo-events workflow=workflow-problem-hm68w time="2022-07-29T12:45:37.474Z" level=info msg="Retry node workflow-problem-hm68w initialized Running" namespace=argo-events workflow=workflow-problem-hm68w time="2022-07-29T12:45:37.474Z" level=info msg="DAG node workflow-problem-hm68w-2520437565 initialized Running" namespace=argo-events workflow=workflow-problem-hm68w time="2022-07-29T12:45:37.474Z" level=info msg="All of node workflow-problem-hm68w(0).checkout dependencies [] completed" namespace=argo-events workflow=workflow-problem-hm68w time="2022-07-29T12:45:37.475Z" level=info msg="Retry node workflow-problem-hm68w-4027359757 initialized Running" namespace=argo-events workflow=workflow-problem-hm68w time="2022-07-29T12:45:37.476Z" level=info msg="Pod node workflow-problem-hm68w-3269103708 initialized Pending" namespace=argo-events workflow=workflow-problem-hm68w time="2022-07-29T12:45:37.502Z" level=info msg="Created pod: workflow-problem-hm68w(0).checkout(0) (workflow-problem-hm68w-3269103708)" namespace=argo-events workflow=workflow-problem-hm68w time="2022-07-29T12:45:37.502Z" level=info msg="TaskSet Reconciliation" namespace=argo-events workflow=workflow-problem-hm68w time="2022-07-29T12:45:37.502Z" level=info msg=reconcileAgentPod namespace=argo-events workflow=workflow-problem-hm68w time="2022-07-29T12:45:37.537Z" level=info msg="Workflow update successful" namespace=argo-events phase=Running resourceVersion=83666971 workflow=workflow-problem-hm68w time="2022-07-29T12:45:47.399Z" level=info msg="Processing workflow" namespace=argo-events workflow=workflow-problem-hm68w time="2022-07-29T12:45:47.400Z" level=info msg="Task-result reconciliation" namespace=argo-events numObjs=0 workflow=workflow-problem-hm68w time="2022-07-29T12:45:47.401Z" level=info msg="node changed" new.message=PodInitializing new.phase=Pending new.progress=0/1 nodeID=workflow-problem-hm68w-3269103708 old.message= old.phase=Pending old.progress=0/1 time="2022-07-29T12:45:47.404Z" level=info msg="TaskSet Reconciliation" namespace=argo-events workflow=workflow-problem-hm68w time="2022-07-29T12:45:47.406Z" level=info msg=reconcileAgentPod namespace=argo-events workflow=workflow-problem-hm68w time="2022-07-29T12:45:47.462Z" level=info msg="Workflow update successful" namespace=argo-events phase=Running resourceVersion=83667061 workflow=workflow-problem-hm68w time="2022-07-29T12:45:57.443Z" level=info msg="Processing workflow" namespace=argo-events workflow=workflow-problem-hm68w time="2022-07-29T12:45:57.445Z" level=info msg="Task-result reconciliation" namespace=argo-events numObjs=1 workflow=workflow-problem-hm68w time="2022-07-29T12:45:57.446Z" level=info msg="task-result changed" namespace=argo-events nodeID=workflow-problem-hm68w-3269103708 workflow=workflow-problem-hm68w time="2022-07-29T12:45:57.446Z" level=info msg="node changed" new.message= new.phase=Running new.progress=0/1 nodeID=workflow-problem-hm68w-3269103708 old.message=PodInitializing old.phase=Pending old.progress=0/1 time="2022-07-29T12:45:57.449Z" level=info msg="TaskSet Reconciliation" namespace=argo-events workflow=workflow-problem-hm68w time="2022-07-29T12:45:57.449Z" level=info msg=reconcileAgentPod namespace=argo-events workflow=workflow-problem-hm68w time="2022-07-29T12:45:57.472Z" level=info msg="Workflow update successful" namespace=argo-events phase=Running resourceVersion=83667135 workflow=workflow-problem-hm68w time="2022-07-29T12:46:07.645Z" level=info msg="Processing workflow" namespace=argo-events workflow=workflow-problem-hm68w time="2022-07-29T12:46:07.647Z" level=info msg="Task-result reconciliation" namespace=argo-events numObjs=1 workflow=workflow-problem-hm68w time="2022-07-29T12:46:07.647Z" level=info msg="node changed" new.message= new.phase=Succeeded new.progress=0/1 nodeID=workflow-problem-hm68w-3269103708 old.message= old.phase=Running old.progress=0/1 time="2022-07-29T12:46:07.648Z" level=info msg="node workflow-problem-hm68w-4027359757 phase Running -> Succeeded" namespace=argo-events workflow=workflow-problem-hm68w time="2022-07-29T12:46:07.649Z" level=info msg="node workflow-problem-hm68w-4027359757 finished: 2022-07-29 12:46:07.649002176 +0000 UTC" namespace=argo-events workflow=workflow-problem-hm68w time="2022-07-29T12:46:07.649Z" level=info msg="All of node workflow-problem-hm68w(0).tag dependencies [checkout] completed" namespace=argo-events workflow=workflow-problem-hm68w time="2022-07-29T12:46:07.649Z" level=info msg="Retry node workflow-problem-hm68w-3152904399 initialized Running" namespace=argo-events workflow=workflow-problem-hm68w time="2022-07-29T12:46:07.650Z" level=info msg="Pod node workflow-problem-hm68w-2551760466 initialized Pending" namespace=argo-events workflow=workflow-problem-hm68w time="2022-07-29T12:46:07.689Z" level=info msg="Created pod: workflow-problem-hm68w(0).tag(0) (workflow-problem-hm68w-2551760466)" namespace=argo-events workflow=workflow-problem-hm68w time="2022-07-29T12:46:07.689Z" level=info msg="TaskSet Reconciliation" namespace=argo-events workflow=workflow-problem-hm68w time="2022-07-29T12:46:07.689Z" level=info msg=reconcileAgentPod namespace=argo-events workflow=workflow-problem-hm68w time="2022-07-29T12:46:07.724Z" level=info msg="Workflow update successful" namespace=argo-events phase=Running resourceVersion=83667212 workflow=workflow-problem-hm68w time="2022-07-29T12:46:12.727Z" level=info msg="cleaning up pod" action=deletePod key=argo-events/workflow-problem-hm68w-3269103708/deletePod time="2022-07-29T12:46:17.694Z" level=info msg="Processing workflow" namespace=argo-events workflow=workflow-problem-hm68w time="2022-07-29T12:46:17.695Z" level=info msg="Task-result reconciliation" namespace=argo-events numObjs=1 workflow=workflow-problem-hm68w time="2022-07-29T12:46:17.696Z" level=info msg="node changed" new.message=PodInitializing new.phase=Pending new.progress=0/1 nodeID=workflow-problem-hm68w-2551760466 old.message= old.phase=Pending old.progress=0/1 time="2022-07-29T12:46:17.698Z" level=info msg="TaskSet Reconciliation" namespace=argo-events workflow=workflow-problem-hm68w time="2022-07-29T12:46:17.698Z" level=info msg=reconcileAgentPod namespace=argo-events workflow=workflow-problem-hm68w time="2022-07-29T12:46:17.741Z" level=info msg="Workflow update successful" namespace=argo-events phase=Running resourceVersion=83667299 workflow=workflow-problem-hm68w time="2022-07-29T12:46:27.735Z" level=info msg="Processing workflow" namespace=argo-events workflow=workflow-problem-hm68w time="2022-07-29T12:46:27.738Z" level=info msg="Task-result reconciliation" namespace=argo-events numObjs=2 workflow=workflow-problem-hm68w time="2022-07-29T12:46:27.738Z" level=info msg="task-result changed" namespace=argo-events nodeID=workflow-problem-hm68w-2551760466 workflow=workflow-problem-hm68w time="2022-07-29T12:46:27.741Z" level=info msg="node changed" new.message="Error (exit code 1)" new.phase=Failed new.progress=0/1 nodeID=workflow-problem-hm68w-2551760466 old.message=PodInitializing old.phase=Pending old.progress=0/1 time="2022-07-29T12:46:27.747Z" level=info msg="1 child nodes of workflow-problem-hm68w(0).tag failed. Trying again..." namespace=argo-events workflow=workflow-problem-hm68w time="2022-07-29T12:46:27.753Z" level=info msg="Pod node workflow-problem-hm68w-873851471 initialized Pending" namespace=argo-events workflow=workflow-problem-hm68w time="2022-07-29T12:46:27.775Z" level=info msg="Created pod: workflow-problem-hm68w(0).tag(1) (workflow-problem-hm68w-873851471)" namespace=argo-events workflow=workflow-problem-hm68w time="2022-07-29T12:46:27.775Z" level=info msg="TaskSet Reconciliation" namespace=argo-events workflow=workflow-problem-hm68w time="2022-07-29T12:46:27.775Z" level=info msg=reconcileAgentPod namespace=argo-events workflow=workflow-problem-hm68w time="2022-07-29T12:46:27.820Z" level=info msg="Workflow update successful" namespace=argo-events phase=Running resourceVersion=83667375 workflow=workflow-problem-hm68w time="2022-07-29T12:46:27.830Z" level=info msg="cleaning up pod" action=labelPodCompleted key=argo-events/workflow-problem-hm68w-2551760466/labelPodCompleted time="2022-07-29T12:46:37.785Z" level=info msg="Processing workflow" namespace=argo-events workflow=workflow-problem-hm68w time="2022-07-29T12:46:37.786Z" level=info msg="Task-result reconciliation" namespace=argo-events numObjs=2 workflow=workflow-problem-hm68w time="2022-07-29T12:46:37.786Z" level=info msg="node changed" new.message=PodInitializing new.phase=Pending new.progress=0/1 nodeID=workflow-problem-hm68w-873851471 old.message= old.phase=Pending old.progress=0/1 time="2022-07-29T12:46:37.790Z" level=info msg="TaskSet Reconciliation" namespace=argo-events workflow=workflow-problem-hm68w time="2022-07-29T12:46:37.790Z" level=info msg=reconcileAgentPod namespace=argo-events workflow=workflow-problem-hm68w time="2022-07-29T12:46:37.841Z" level=info msg="Workflow update successful" namespace=argo-events phase=Running resourceVersion=83667451 workflow=workflow-problem-hm68w time="2022-07-29T12:46:47.845Z" level=info msg="Processing workflow" namespace=argo-events workflow=workflow-problem-hm68w time="2022-07-29T12:46:47.846Z" level=info msg="Task-result reconciliation" namespace=argo-events numObjs=3 workflow=workflow-problem-hm68w time="2022-07-29T12:46:47.846Z" level=info msg="task-result changed" namespace=argo-events nodeID=workflow-problem-hm68w-873851471 workflow=workflow-problem-hm68w time="2022-07-29T12:46:47.846Z" level=info msg="node changed" new.message="Error (exit code 1)" new.phase=Failed new.progress=0/1 nodeID=workflow-problem-hm68w-873851471 old.message=PodInitializing old.phase=Pending old.progress=0/1 time="2022-07-29T12:46:47.849Z" level=info msg="2 child nodes of workflow-problem-hm68w(0).tag failed. Trying again..." namespace=argo-events workflow=workflow-problem-hm68w time="2022-07-29T12:46:47.850Z" level=info msg="Pod node workflow-problem-hm68w-3021827988 initialized Pending" namespace=argo-events workflow=workflow-problem-hm68w time="2022-07-29T12:46:47.890Z" level=info msg="Created pod: workflow-problem-hm68w(0).tag(2) (workflow-problem-hm68w-3021827988)" namespace=argo-events workflow=workflow-problem-hm68w time="2022-07-29T12:46:47.890Z" level=info msg="TaskSet Reconciliation" namespace=argo-events workflow=workflow-problem-hm68w time="2022-07-29T12:46:47.891Z" level=info msg=reconcileAgentPod namespace=argo-events workflow=workflow-problem-hm68w time="2022-07-29T12:46:47.949Z" level=info msg="Workflow update successful" namespace=argo-events phase=Running resourceVersion=83667531 workflow=workflow-problem-hm68w time="2022-07-29T12:46:47.958Z" level=info msg="cleaning up pod" action=labelPodCompleted key=argo-events/workflow-problem-hm68w-873851471/labelPodCompleted time="2022-07-29T12:46:57.889Z" level=info msg="Processing workflow" namespace=argo-events workflow=workflow-problem-hm68w time="2022-07-29T12:46:57.890Z" level=info msg="Task-result reconciliation" namespace=argo-events numObjs=3 workflow=workflow-problem-hm68w time="2022-07-29T12:46:57.890Z" level=info msg="node changed" new.message= new.phase=Running new.progress=0/1 nodeID=workflow-problem-hm68w-3021827988 old.message= old.phase=Pending old.progress=0/1 time="2022-07-29T12:46:57.892Z" level=info msg="TaskSet Reconciliation" namespace=argo-events workflow=workflow-problem-hm68w time="2022-07-29T12:46:57.892Z" level=info msg=reconcileAgentPod namespace=argo-events workflow=workflow-problem-hm68w time="2022-07-29T12:46:57.934Z" level=info msg="Workflow update successful" namespace=argo-events phase=Running resourceVersion=83667604 workflow=workflow-problem-hm68w time="2022-07-29T12:47:07.931Z" level=info msg="Processing workflow" namespace=argo-events workflow=workflow-problem-hm68w time="2022-07-29T12:47:07.932Z" level=info msg="Task-result reconciliation" namespace=argo-events numObjs=4 workflow=workflow-problem-hm68w time="2022-07-29T12:47:07.932Z" level=info msg="task-result changed" namespace=argo-events nodeID=workflow-problem-hm68w-3021827988 workflow=workflow-problem-hm68w time="2022-07-29T12:47:07.932Z" level=info msg="node changed" new.message="Error (exit code 1)" new.phase=Failed new.progress=0/1 nodeID=workflow-problem-hm68w-3021827988 old.message= old.phase=Running old.progress=0/1 time="2022-07-29T12:47:07.937Z" level=info msg="3 child nodes of workflow-problem-hm68w(0).tag failed. Trying again..." namespace=argo-events workflow=workflow-problem-hm68w time="2022-07-29T12:47:07.938Z" level=info msg="Pod node workflow-problem-hm68w-2417686609 initialized Pending" namespace=argo-events workflow=workflow-problem-hm68w time="2022-07-29T12:47:08.003Z" level=info msg="Created pod: workflow-problem-hm68w(0).tag(3) (workflow-problem-hm68w-2417686609)" namespace=argo-events workflow=workflow-problem-hm68w time="2022-07-29T12:47:08.003Z" level=info msg="TaskSet Reconciliation" namespace=argo-events workflow=workflow-problem-hm68w time="2022-07-29T12:47:08.003Z" level=info msg=reconcileAgentPod namespace=argo-events workflow=workflow-problem-hm68w time="2022-07-29T12:47:08.070Z" level=info msg="Workflow update successful" namespace=argo-events phase=Running resourceVersion=83667681 workflow=workflow-problem-hm68w time="2022-07-29T12:47:08.079Z" level=info msg="cleaning up pod" action=labelPodCompleted key=argo-events/workflow-problem-hm68w-3021827988/labelPodCompleted time="2022-07-29T12:47:17.992Z" level=info msg="Processing workflow" namespace=argo-events workflow=workflow-problem-hm68w time="2022-07-29T12:47:17.993Z" level=info msg="Task-result reconciliation" namespace=argo-events numObjs=4 workflow=workflow-problem-hm68w time="2022-07-29T12:47:17.994Z" level=info msg="node changed" new.message= new.phase=Running new.progress=0/1 nodeID=workflow-problem-hm68w-2417686609 old.message= old.phase=Pending old.progress=0/1 time="2022-07-29T12:47:17.996Z" level=info msg="TaskSet Reconciliation" namespace=argo-events workflow=workflow-problem-hm68w time="2022-07-29T12:47:17.996Z" level=info msg=reconcileAgentPod namespace=argo-events workflow=workflow-problem-hm68w time="2022-07-29T12:47:18.045Z" level=info msg="Workflow update successful" namespace=argo-events phase=Running resourceVersion=83667766 workflow=workflow-problem-hm68w time="2022-07-29T12:47:28.033Z" level=info msg="Processing workflow" namespace=argo-events workflow=workflow-problem-hm68w time="2022-07-29T12:47:28.034Z" level=info msg="Task-result reconciliation" namespace=argo-events numObjs=5 workflow=workflow-problem-hm68w time="2022-07-29T12:47:28.034Z" level=info msg="task-result changed" namespace=argo-events nodeID=workflow-problem-hm68w-2417686609 workflow=workflow-problem-hm68w time="2022-07-29T12:47:28.034Z" level=info msg="node changed" new.message="Error (exit code 1)" new.phase=Failed new.progress=0/1 nodeID=workflow-problem-hm68w-2417686609 old.message= old.phase=Running old.progress=0/1 time="2022-07-29T12:47:28.042Z" level=info msg="No more retries left. Failing..." namespace=argo-events workflow=workflow-problem-hm68w time="2022-07-29T12:47:28.042Z" level=info msg="node workflow-problem-hm68w-3152904399 phase Running -> Failed" namespace=argo-events workflow=workflow-problem-hm68w time="2022-07-29T12:47:28.042Z" level=info msg="node workflow-problem-hm68w-3152904399 message: No more retries left" namespace=argo-events workflow=workflow-problem-hm68w time="2022-07-29T12:47:28.042Z" level=info msg="node workflow-problem-hm68w-3152904399 finished: 2022-07-29 12:47:28.042507727 +0000 UTC" namespace=argo-events workflow=workflow-problem-hm68w time="2022-07-29T12:47:28.042Z" level=info msg="Outbound nodes of workflow-problem-hm68w-2520437565 set to [workflow-problem-hm68w-2417686609]" namespace=argo-events workflow=workflow-problem-hm68w time="2022-07-29T12:47:28.042Z" level=info msg="node workflow-problem-hm68w-2520437565 phase Running -> Failed" namespace=argo-events workflow=workflow-problem-hm68w time="2022-07-29T12:47:28.042Z" level=info msg="node workflow-problem-hm68w-2520437565 finished: 2022-07-29 12:47:28.04273348 +0000 UTC" namespace=argo-events workflow=workflow-problem-hm68w time="2022-07-29T12:47:28.042Z" level=info msg="Checking daemoned children of workflow-problem-hm68w-2520437565" namespace=argo-events workflow=workflow-problem-hm68w time="2022-07-29T12:47:28.044Z" level=info msg="1 child nodes of workflow-problem-hm68w failed. Trying again..." namespace=argo-events workflow=workflow-problem-hm68w time="2022-07-29T12:47:28.044Z" level=info msg="DAG node workflow-problem-hm68w-2856137040 initialized Running" namespace=argo-events workflow=workflow-problem-hm68w time="2022-07-29T12:47:28.044Z" level=info msg="All of node workflow-problem-hm68w(1).checkout dependencies [] completed" namespace=argo-events workflow=workflow-problem-hm68w time="2022-07-29T12:47:28.046Z" level=info msg="Retry node workflow-problem-hm68w-372661854 initialized Running" namespace=argo-events workflow=workflow-problem-hm68w time="2022-07-29T12:47:28.047Z" level=info msg="Pod node workflow-problem-hm68w-24444429 initialized Pending" namespace=argo-events workflow=workflow-problem-hm68w time="2022-07-29T12:47:28.100Z" level=info msg="Created pod: workflow-problem-hm68w(1).checkout(0) (workflow-problem-hm68w-24444429)" namespace=argo-events workflow=workflow-problem-hm68w time="2022-07-29T12:47:28.100Z" level=info msg="TaskSet Reconciliation" namespace=argo-events workflow=workflow-problem-hm68w time="2022-07-29T12:47:28.100Z" level=info msg=reconcileAgentPod namespace=argo-events workflow=workflow-problem-hm68w time="2022-07-29T12:47:28.154Z" level=info msg="Workflow update successful" namespace=argo-events phase=Running resourceVersion=83667844 workflow=workflow-problem-hm68w time="2022-07-29T12:47:28.170Z" level=info msg="cleaning up pod" action=labelPodCompleted key=argo-events/workflow-problem-hm68w-2417686609/labelPodCompleted time="2022-07-29T12:47:38.101Z" level=info msg="Processing workflow" namespace=argo-events workflow=workflow-problem-hm68w time="2022-07-29T12:47:38.101Z" level=info msg="Task-result reconciliation" namespace=argo-events numObjs=5 workflow=workflow-problem-hm68w time="2022-07-29T12:47:38.102Z" level=info msg="node changed" new.message=PodInitializing new.phase=Pending new.progress=0/1 nodeID=workflow-problem-hm68w-24444429 old.message= old.phase=Pending old.progress=0/1 time="2022-07-29T12:47:38.104Z" level=info msg="TaskSet Reconciliation" namespace=argo-events workflow=workflow-problem-hm68w time="2022-07-29T12:47:38.104Z" level=info msg=reconcileAgentPod namespace=argo-events workflow=workflow-problem-hm68w time="2022-07-29T12:47:38.149Z" level=info msg="Workflow update successful" namespace=argo-events phase=Running resourceVersion=83667921 workflow=workflow-problem-hm68w time="2022-07-29T12:47:49.070Z" level=info msg="Processing workflow" namespace=argo-events workflow=workflow-problem-hm68w time="2022-07-29T12:47:49.072Z" level=info msg="Task-result reconciliation" namespace=argo-events numObjs=5 workflow=workflow-problem-hm68w time="2022-07-29T12:47:49.072Z" level=info msg="node changed" new.message="Error (exit code 1): rename /mainctrfs/workspace/project.tmp /mainctrfs/workspace/project: file exists" new.phase=Error new.progress=0/1 nodeID=workflow-problem-hm68w-24444429 old.message=PodInitializing old.phase=Pending old.progress=0/1 time="2022-07-29T12:47:49.075Z" level=info msg="Node not set to be retried after status: Error" namespace=argo-events workflow=workflow-problem-hm68w time="2022-07-29T12:47:49.075Z" level=info msg="node workflow-problem-hm68w-372661854 phase Running -> Error" namespace=argo-events workflow=workflow-problem-hm68w time="2022-07-29T12:47:49.076Z" level=info msg="node workflow-problem-hm68w-372661854 message: Error (exit code 1): rename /mainctrfs/workspace/project.tmp /mainctrfs/workspace/project: file exists" namespace=argo-events workflow=workflow-problem-hm68w time="2022-07-29T12:47:49.076Z" level=info msg="node workflow-problem-hm68w-372661854 finished: 2022-07-29 12:47:49.076487529 +0000 UTC" namespace=argo-events workflow=workflow-problem-hm68w time="2022-07-29T12:47:49.076Z" level=info msg="Skipped node workflow-problem-hm68w-961798914 initialized Omitted (message: omitted: depends condition not met)" namespace=argo-events workflow=workflow-problem-hm68w time="2022-07-29T12:47:49.076Z" level=info msg="Outbound nodes of workflow-problem-hm68w-2856137040 set to [workflow-problem-hm68w-961798914]" namespace=argo-events workflow=workflow-problem-hm68w time="2022-07-29T12:47:49.077Z" level=info msg="node workflow-problem-hm68w-2856137040 phase Running -> Error" namespace=argo-events workflow=workflow-problem-hm68w time="2022-07-29T12:47:49.077Z" level=info msg="node workflow-problem-hm68w-2856137040 finished: 2022-07-29 12:47:49.077086018 +0000 UTC" namespace=argo-events workflow=workflow-problem-hm68w time="2022-07-29T12:47:49.077Z" level=info msg="Checking daemoned children of workflow-problem-hm68w-2856137040" namespace=argo-events workflow=workflow-problem-hm68w time="2022-07-29T12:47:49.078Z" level=info msg="Node not set to be retried after status: Error" namespace=argo-events workflow=workflow-problem-hm68w time="2022-07-29T12:47:49.078Z" level=info msg="node workflow-problem-hm68w phase Running -> Error" namespace=argo-events workflow=workflow-problem-hm68w time="2022-07-29T12:47:49.078Z" level=info msg="node workflow-problem-hm68w finished: 2022-07-29 12:47:49.078742993 +0000 UTC" namespace=argo-events workflow=workflow-problem-hm68w time="2022-07-29T12:47:49.078Z" level=info msg="TaskSet Reconciliation" namespace=argo-events workflow=workflow-problem-hm68w time="2022-07-29T12:47:49.079Z" level=info msg=reconcileAgentPod namespace=argo-events workflow=workflow-problem-hm68w time="2022-07-29T12:47:49.079Z" level=info msg="Updated phase Running -> Error" namespace=argo-events workflow=workflow-problem-hm68w time="2022-07-29T12:47:49.079Z" level=info msg="Marking workflow completed" namespace=argo-events workflow=workflow-problem-hm68w time="2022-07-29T12:47:49.079Z" level=info msg="Marking workflow as pending archiving" namespace=argo-events workflow=workflow-problem-hm68w time="2022-07-29T12:47:49.079Z" level=info msg="Checking daemoned children of " namespace=argo-events workflow=workflow-problem-hm68w time="2022-07-29T12:47:49.085Z" level=info msg="cleaning up pod" action=deletePod key=argo-events/workflow-problem-hm68w-1340600742-agent/deletePod time="2022-07-29T12:47:49.138Z" level=info msg="Workflow update successful" namespace=argo-events phase=Error resourceVersion=83668007 workflow=workflow-problem-hm68w time="2022-07-29T12:47:49.372Z" level=info msg="archiving workflow" namespace=argo-events uid=e9a595ab-2412-4677-a413-507227dfb8e2 workflow=workflow-problem-hm68w time="2022-07-29T12:47:49.377Z" level=info msg="cleaning up pod" action=labelPodCompleted key=argo-events/workflow-problem-hm68w-24444429/labelPodCompleted time="2022-07-29T12:47:49.497Z" level=info msg="Queueing Error workflow argo-events/workflow-problem-hm68w for delete in 2m0s due to TTL" time="2022-07-29T12:49:50.000Z" level=info msg="Deleting garbage collected workflow 'argo-events/workflow-problem-hm68w'" time="2022-07-29T12:49:50.028Z" level=info msg="Successfully deleted 'argo-events/workflow-problem-hm68w'" ``` ---

Message from the maintainers:

Impacted by this bug? Give it a 👍. We prioritise the issues with the most 👍.

stale[bot] commented 2 years ago

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. If this is a mentoring request, please provide an update here. Thank you for your contributions.

rgeraskin commented 2 years ago

confirm, this bug still exists

stale[bot] commented 2 years ago

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. If this is a mentoring request, please provide an update here. Thank you for your contributions.

rgeraskin commented 2 years ago

confirm, this bug still exists

sarabala1979 commented 2 years ago

@rgeraskin can you add an exit handler on the step?

rgeraskin commented 2 years ago

@sarabala1979 sorry I can't figure it out how it will help me? Maybe you are talking about deleting the content manually by handler?

stale[bot] commented 2 years ago

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. If this is a mentoring request, please provide an update here. Thank you for your contributions.

tooptoop4 commented 4 days ago

likely one of the rename commands in https://github.com/argoproj/argo-workflows/blob/f470fdab279a4e0f28c6e324f1a337dafb73ec13/workflow/executor/executor.go to change