Build stage runs for a long time, marked successful after build fails

[spadgett]

ruby-sample-build-4 was kicked off by sample-pipeline-4. The s2i build failed after one second, but the pipeline stage ran for ~5 minutes.

Much later the pipeline stage was marked as successful even when the build failed.

Here is how I'm running the build from my Jenkinsfile

  final project = 'myproject'
  final bc = 'ruby-sample-build'
  final dc = 'frontend'

  stage 'Build'
  def builder = new com.openshift.jenkins.plugins.pipeline.OpenShiftBuilder("", bc, project, "", "", "", "", "true", "", "")
  step builder

I can see the build was failed in the Jenkins log.

Exiting "Trigger OpenShift Build" unsuccessfully; build "ruby-sample-build-4" has completed with status: [Failed].

@gabemontero Will the DSL changes correctly mark the stage as failed when the build fails? https://trello.com/c/jOEuMkze

/cc @bparees @jwforres

[spadgett]

/cc @sspeiche

[bparees]

is that the entirety of your jenkinsfile? i know i saw the same issue where a failing build step didn't immediately fail the entire pipeline, but i didn't see it hang for 5 minutes, it just completed the pipeline. Any idea what it was doing for the rest of the time?

[gabemontero]

@spadgett this had been reported by an upstream customer, and he at least had confirmed that https://github.com/openshift/jenkins-plugin/commit/d0481f7fb63ed27bf8d3f38370cf7b80abbb0331 had addressed the issue for him (so this is independent from / unrelated to the DSL changes).

Can you first confirm the level of the plugin you are running with?

[spadgett]

is that the entirety of your jenkinsfile?

No, I have a deploy step after the build completes.

Can you first confirm the level of the plugin you are running with?

1.0.10 now. I can try 1.0.12.

[gabemontero]

Yeah, the fix went into 1.0.11; if for some reason 1.0.12 does not solve it for you, I'll dive into how to debug.

[spadgett]

@gabemontero I think there are two problems here

1. The build in Jenkins takes a long time to fail
2. The build stage is marked as successful in the pipeline

1.0.12 solves (1), but not (2). The stage in the pipeline is successful even though the s2i build failed

@bparees I was able to reproduce this with your pipeline templates by editing the bc/ruby-sample-build to use a bogus source URL

[gabemontero]

@spadgett can you show me the console output?

[spadgett]

Jenkins console log:

OpenShift Build pipelineproject/sample-pipeline-2 from https://github.com/openshift/ruby-hello-world
OpenShift Build pipelineproject/sample-pipeline-2 from https://github.com/openshift/ruby-hello-world
[Pipeline] Allocate node : Start
Running on 79f36425cb3 in /home/jenkins/workspace/sample-pipeline
[Pipeline] node {
[Pipeline] stage (build)
Entering stage build
Proceeding
[Pipeline] step

Starting the "Trigger OpenShift Build" step with build config "ruby-sample-build" from the project "pipelineproject".
  Started build "ruby-sample-build-2" and waiting for build completion ...
Build logs:  
W0524 15:06:27.388433       1 builder.go:55] Master version "v1.3.0-alpha.0-660-g5f098ef-dirty" does not match Builder image version "v1.3.0-alpha.0"
I0524 15:06:27.571288       1 source.go:197] Downloading "https://github.com/openshift/ruby-hello-world.git.bogus" ...
F0524 15:06:28.288366       1 builder.go:204] Error: build error: failed to fetch requested repository "https://github.com/openshift/ruby-hello-world.git.bogus" with provided credentials

Exiting "Trigger OpenShift Build" unsuccessfully; build "ruby-sample-build-2" has completed with status:  [Failed].
[Pipeline] stage (deploy)
Entering stage deploy
Proceeding
[Pipeline] step

Starting "Trigger OpenShift Deployment" with deployment config "frontend" from the project "pipelineproject".

Exiting "Trigger OpenShift Deployment" successfully; deployment "frontend-2" has completed with status:  [Complete].
[Pipeline] } //node
[Pipeline] Allocate node : End
[Pipeline] End of Pipeline
Finished: SUCCESS

All pipeline stages are succesful:

Here is my Jenkinsfile (from @bparees pipeline template https://github.com/openshift/origin/pull/8973)

node('agent') {
stage 'build'
def builder = new com.openshift.jenkins.plugins.pipeline.OpenShiftBuilder("", "ruby-sample-build", "pipelineproject", "", "", "", "", "true", "", "")
step builder
stage 'deploy'
def deployer = new com.openshift.jenkins.plugins.pipeline.OpenShiftDeployer("","frontend","pipelineproject","","")
step deployer
}

[bparees]

so the issue is either: 1) the openshift plugin isn't reporting that the step failed 2) the jenkins pipeline plugin doesn't actually fail the pipeline when a step fails.

i don't know enough about either to speculate where the problem actually is.

[gabemontero]

@spadgett thanks, I believe I know what is going on now.

Right after the message

Exiting "Trigger OpenShift Build" unsuccessfully; build "ruby-sample-build-2" has completed with status: [Failed].

openshift-pipeline plugin returns false. In the classic Jenkins job, that will result in the job failing. The Jenkins API in that case expects a boolean it will interpret.

However, in the very similar, yet different, class hierarchy for the pipeline stuff, the same api has avoid return, and they appear to want you to throw a IOException or AbortException (the javadoc says that is the "polite" exception :-) ) if there was a problem.

I'll use this issue to add that logic (separate from the DSL stuff).

[bparees]

but apparently @gabemontero does :)

[gabemontero]

@spadgett try installing this version of the plugin

[gabemontero]

Here is the associated commit

[spadgett]

Works great. Thanks @gabemontero

ERROR: The OpenShift Build Step com.openshift.jenkins.plugins.pipeline.OpenShiftBaseStep was unsuccessful

I'd suggest changing the message if possible.

[gabemontero]

Cool deal. Changes to the message are possible - do you have a specific suggestion for the message?

On Tue, May 24, 2016 at 1:56 PM, Sam Padgett notifications@github.com wrote:

Works great. Thanks @gabemontero https://github.com/gabemontero

ERROR: The OpenShift Build Step com.openshift.jenkins.plugins.pipeline.OpenShiftBaseStep was unsuccessful

I'd suggest changing the message if possible.

— You are receiving this because you were mentioned. Reply to this email directly or view it on GitHub https://github.com/openshift/jenkins-plugin/issues/41#issuecomment-221347239

[spadgett]

I think just "ERROR: OpenShift build failed." or "ERROR: OpenShift deployment failed." etc. are more clear.

[gabemontero]

OK, I went with our "meticulously" crafted "official" step titles :-) ... i.e.:

ERROR: "Trigger OpenShift Build" failed

Going ahead and marking this closed.