reanahub / reana

REANA: Reusable research data analysis platform
https://docs.reana.io
MIT License
127 stars 54 forks source link

cli: failed to stop workflow #266

Closed diegodelemos closed 4 years ago

diegodelemos commented 4 years ago

When trying to stop a running workflow:

$ # roofit example
$ git diff                                                                                                                                                      [5:00:56]
diff --git a/reana.yaml b/reana.yaml                                                                                                                                                                                                   
index e1355ca..51be33c 100644                                                                                      
--- a/reana.yaml                                                                                                   
+++ b/reana.yaml                                                                                                                                                                                                                       
@@ -19,7 +19,7 @@ workflow:                           
       - name: fitdata                                
         environment: 'reanahub/reana-env-root6:6.18.04'
         commands:                                                                                                                                                                                                                     
-        - root -b -q 'code/fitdata.C("${data}","${plot}")'                                                                                                                                                                            
+        - sleep 1000                              
 outputs:                                             
   files:                                             
     - results/plot.png 
$ reana-client run
[INFO] Creating a workflow...                         
workflow.19                                           
[INFO] Uploading files...                             
File code/gendata.C was successfully uploaded.        
File code/fitdata.C was successfully uploaded.                                                                                                                                                                                         
[INFO] Starting workflow...                                                                                                                                                                                                            
workflow.19 is running 

Then first workflow job finished (14e3e86f-31ac-4990-9405-1f888d 5adb98) and the second one (70c6e3eb-6142-43fb-8169-aa90c060023c-4r2dn) starts to run:

$ kubectl get pods                                                                                                                                                      [5:01:49]
NAME                                                            READY   STATUS    RESTARTS   AGE
70c6e3eb-6142-43fb-8169-aa90c060023c-4r2dn                      1/1     Running   0          9s
reana-batch-serial-dcac5ed9-1d93-4a91-8631-ce518d4f8be2-l2gcg   2/2     Running   0          34s
reana-cache-88b76b854-85ptz                                     1/1     Running   0          36m
reana-db-5cff5946c7-xwmtk                                       1/1     Running   0          36m
reana-message-broker-65bbb5956f-7mgss                           1/1     Running   0          36m
reana-server-6487dc6d74-mkrcc                                   2/2     Running   0          32m
reana-traefik-86556f5678-h9nqk                                  1/1     Running   0          36m
reana-ui-597887756f-hhhsd                                       1/1     Running   0          36m
reana-wdb-574d66ff44-6kdhk                                      1/1     Running   0          36m
reana-workflow-controller-64d7cd56b4-4cq9q                      2/2     Running   0          34m

User tries to stop the workflow execution:

$ reana-client stop -w workflow.19 --force                                                                                                                          [5:01:45]
Workflow could not be stopped: 
(404)
Reason: Not Found
HTTP response headers: HTTPHeaderDict({'Cache-Control': 'no-cache, private', 'Content-Type': 'application/json', 'Date': 'Fri, 07 Feb 2020 16:01:49 GMT', 'Content-Length': '262'})
HTTP response body: {"kind":"Status","apiVersion":"v1","metadata":{},"status":"Failure","message":"jobs.batch \"14e3e86f-31ac-4990-9405-1f888d5adb98\" not found","reason":"NotFound","details":{"name":"14e3e86f-31ac-4990-9405-1f888d
5adb98","group":"batch","kind":"jobs"},"code":404}

We should:

diegodelemos commented 4 years ago

This is also happening when jobs are stuck in ImagePullBackoff, see conversation and complete issue about ImagePullBackoff problem.