cache invalidation and reuse troubles

[tiborsimko]

A workflow execution finished successfully but did not return any results. The problem is connected to job cache handling, the system cannot copy job cache output.

$ reana-client status -w workflow.14
NAME       RUN_NUMBER   CREATED               STATUS     PROGRESS
workflow   14           2019-02-03T21:47:05   finished   1/1

$ reana-client list -w workflow.14
NAME                                                                SIZE    LAST-MODIFIED
code/worldpopulation.ipynb                                          19221   2019-02-03T21:47:06
data/World_historical_and_predicted_populations_in_percentage.csv   574     2019-02-03T21:47:06

$ kl batch-serial-21f42a60-f517-4eb8-a6aa-43e29c235ebc-xxk67
cp: cannot stat '/var/reana/users/4a3bdfcc-201c-48ec-94ec-9ad2bd5e3b63/workflows/archive/e70c5a17-a368-4c55-bff2-5ec4d9571bde/*': No such file or directory
~~~~~ Copied from cache

(1) This could have happened due to past workflow deletions. I have deleted quite a few run tests before the demo, using many --include-... options. We should check the workflow deletion code whether everything is properly invalidated including cache and friends.

(2) Additionally, the workflow run should not end with "success" status in case of internal errors of this kind. We should rather change to "failed". This problem can be tackled first before investigating the former.

[roksys]

Probably it was caused by running similar command.

(reana-client3) rokas@rokas reana-demo-helloworld (master) $ reana-client delete --include-all-runs --include-workspace --include-records  -w workflow.29
Workflow could not be deleted: 
resource 'users/00000000-0000-0000-0000-000000000000/workflows/18337c79-5646-4f43-b7be-868c8f180b76/../archive/253d0c83-eef1-4d62-a5a1-bf8ffe1d7296' not found

Workspace got deleted, but not cache entry in DB.