Failure processing step

[igor-ivanov]

Feature request:

It would be nice to have special step that is executed during failure when job stopes processing next steps. This ability can be used in following case when job should collect and publish artifacts just at the end.

• It can be done currently additng special final step but it is not called during failure on one of the previous steps.
• onfail does not help because it can describe commands but can not publish using archiveArtifacts

[mike-dubman]

pipeline_stop is always executed and it has env variable indicating succ/fail, plz see example here: https://github.com/Mellanox/ci-demo/blob/master/.ci/examples/job_matrix_containerSelector.yaml#L45 archiveArtifacts can be used globally, per step, per pipeline-stop/start special steps as well

does it help?

[igor-ivanov]

pipeline_stop is always executed and it has env variable indicating succ/fail, plz see example here: https://github.com/Mellanox/ci-demo/blob/master/.ci/examples/job_matrix_containerSelector.yaml#L45 archiveArtifacts can be used globally, per step, per pipeline-stop/start special steps as well

does it help?

It does not help because pipeline_stop is executed on master node when results can be collected from different $WORKSPACE. For example I found that $WORKSPACE can point on different locations as to /home/jenkins/agent/workspace/LIBVMA-PRO in steps but to /scrap/jenkins/workspace/LIBVMA-PRO when I use $WORKSPACE inside pipeline_stop.

[mike-dubman]

attachArtifact (attachJunit, attachTap) can be per step as well and always executed (in step $WS) regardless if step failed or succeeded

[igor-ivanov]

attachArtifact (attachJunit, attachTap) can be per step as well and always executed (in step $WS) regardless if step failed or succeeded

It is clear. But I do not need to publishng artifacts in success on intermidiate step. Just as last step. And on failure when I know that last step is not reached I would like to collect artifacts. I would like to post artifatcs just once on failure step or as last step

[mike-dubman]

you can add archiveTap to last step only if step-1 fails, step.archiveTap will not take effect

[igor-ivanov]

you can add archiveTap to last step only if step-1 fails, step.archiveTap will not take effect

Yep. It is a reason that this feature request was submitted

[mike-dubman]

cool. can be closed? or I missed smth

[igor-ivanov]

cool. can be closed? or I missed smth

No. I would like to post artifatcs just once if on failure step or as last step. Current implementation does not allow to do what I need.

1. I can set special step to submit artifacts as last step but in case failure on previous step it is not executed. As a result artifacts are not published
2. I can set publishing artifacts for each of steps and results will be published for every steps on success or on failure. But I do not want to submit results on success.

[mike-dubman]

• multiple pipelines (per dimension) running in parallel (each in own containers and WS) and not aware of each other.
• when job fails - it is known in "pipeline_stop" with pipeline_status var set and in its own WS.
• why you need such distinction (attach only on fail). It can be "always attach" and on fail you just will check attachments.

[igor-ivanov]

Existing project jenkins scripts supports test results collection internally in tar-file step by step and get all results in single file after failure or success comppletion. Getting such approach allows to use this locally w/o using jenkins and use in previous jenkins scheme. Submitting this tar-file on every step from ci-demo means getting mostly the same tar-file many times.

[mike-dubman]

per step, one can use onfail and always directive to run commands. if we expand onfail/always to specify attachXXX - will it do it?

steps:

- name: step1
   run: command
   onfail:
       archiveTap: '**/*.tap'

WDYT

[igor-ivanov]

This approach can solve my issue but has minor weakness. I think ArchiveTap and archiveArtifacts should be supported unde onfail

Weakness: In this case I must duplicate the same code for all steps. There is a chance that doing modifications in one part a user can forget to repate it for all places.

For example I should duplicate:

    run: |
      if [ -d ${WORKSPACE}/jenkins ]; then
        gzip ${WORKSPACE}/jenkins/*.tar 2>/dev/null || true
        pushd ${WORKSPACE}/jenkins/ ;
          for f in *.tar.gz ; do [ -e "$f" ] && mv "$f" "${name}-$f" ; done ;
        popd
        pushd ${WORKSPACE}/jenkins/${flags};
          for f in *.tap ; do [ -e "$f" ] && mv "$f" "${BUILD_NUMBER}-${name}-${flags}-$f" ; done ;
        popd
      fi
    archiveArtifacts: |
      jenkins/*tar.gz,
      jenkins/${flags}/*.tap
    archiveTap: |
      **/**/*.tap

[mike-dubman]

@igor-ivanov - please see if following can be helpful: https://github.com/Mellanox/ci-demo/blob/master/.ci/examples/job_matrix_onfail.yaml#L32

added "-onfail" modifier support for archiveXXX command

[igor-ivanov]

Thanks