search

reanahub / reana-workflow-engine-snakemake

REANA Workflow Engine Snakemake
MIT License
0 stars 22 forks source link

executor: add kubernetes_job_timeout parameter #32

Closed VMois closed 2 years ago

VMois commented 2 years ago

closes reanahub/reana-job-controller#343

How to test:

This PR is part of a bigger group of PRs. Please refer to the addressed issue to see all of them.

  1. Modify Snakemake helloworld example, with kubernetes_job_timeout=20 (in seconds) and sleeptime=1 (1 second so workflow will definitely run longer than 20 seconds):

workflow/snakemake/inputs.yaml

sleeptime: 1  # <= here
helloworld: code/helloworld.py
inputfile: data/names.txt

workflow/snakemake/Snakemake

...
rule helloworld:
   ...
    container:
        "docker://python:2.7-slim"
    resources:
        kubernetes_job_timeout=20  # <= here
    shell:
    ...

  1. Start workflow reana-client run -w snake-timeout -f reana-snakemake.yaml

  2. After 30 seconds, check using reana-client list if workflow failed.

  3. Using reana-client logs -w snakemake-timeout check if the reason is Job was killed due to timeout. at the end.

  4. Try to input string or object instead of integer for kubernetes_job_timeout, the workflow should fail and have logs indicating that. This will not work right now due to this issue. It will be fixed after this PR.

VMois commented 2 years ago

the limit/validation errors are not visible from the UI

Validation errors are visible in workflow engine logs, but, you are right, not in job logs. If we want to include error logs into the job we will need to update r-job-controller, I guess. Unless we want to show workflow engine logs in UI.

tiborsimko commented 2 years ago

Bumping priority of reanahub/reana-ui#166 as the limit/validation errors are not visible from the UI. Should we include this in 0.8.1? cc/ @tiborsimko

Considering that we won't work on the Live-Logs sprint in the near future, I agree.. I moved this to 0.8.x kanban board backlog, so that we can take it for 0.8.2 for example.