start build with -F option doesn't print logs

[GolanTrevize10]

I'm using plugin version 0.9.6 with Jenkins 2.74. $ oc version oc v3.6.0+c4dd4cf kubernetes v1.6.1+5115d708d7 features: Basic-Auth

Server xxxxxxxxxxxxx openshift v3.6.173.0.5 kubernetes v1.6.1+5115d708d7

When I try to start build from Jenkins using the -F flag using this command def bc = openshift.selector("bc/jws-app") bc.startBuild('--follow') or bc.startBuild('-F)

Jenkins waits until build is finished but it does not print the logs in the console. If I start the build from command line oc start-build bc/jws-app -F

then it works correctly and I can see the logs in the console

[gabemontero]

Hi (again :-) ) @GolanTrevize10

yeah, currently, this plugin's startBuild verb does not make the necessary changes in our interactions with the durable task plugin to stream the build logs in real time.

Short term, you can get the streaming behavior by removing the --follow and -F from your startBuild, and employ bc.logs('-f') .... see https://github.com/openshift/jenkins-client-plugin#actions-speak-louder-than-words

That said, @jupierce and I talked a bit, and allowing streaming logs to startBuild could be fairly clean and still in line with how we'd like to the fluent style syntax of this plugin to evolve. I'll see if I can construct a quick patch in the next day or two for you to try; otherwise, I'll post a new requirement on our trello board.

@bparees fyi

[gabemontero]

yep ... i've got something working ... pull the plugin from this zip file, install it, and see how this works for you
client-plugin.zip

[jupierce]

@GolanTrevize10 generally, all invocations of 'oc' by the plugin result in capturing the stdout and returning it to you in the resulting groovy object. This stdout/stderr is not generally streamed to the Jenkins console because it would be quite noisy. In this example:

result = bc.startBuild('--follow')

After invoking this, the 'result' variable would contain the stdout (along with other information like return code, command invoked, etc). You can see the fields of this return object in the console: echo "${result}"

See https://github.com/openshift/jenkins-client-plugin/blob/master/src/main/resources/com/openshift/jenkins/plugins/pipeline/OpenShiftGlobalVariable/help.jelly#L498 for information about all the fields in this object (this help is rendered inside of Jenkins in the Global Variable help of Pipeline Syntax).

.logs(...) is a special API which, since we assume the only reason to call it is because you want to print the logs in the console, goes ahead and streams its output to Jenkins for you.

[gabemontero]

@GolanTrevize10 - I've merged --follow and -F support for startBuild just now. I'll cut a new version of the plugin later today.

[gabemontero]

ok the release cutting has been submitted ... look for version 0.9.7 at the jenkins download center ... typically it shows up in a couple of hours after I've submitted

[scoheb]

Hi, I am getting an error when using --follow:

unexpected stream type ""

This causes the startBuild command to fail the build.

Here is the console log snippet: https://paste.fedoraproject.org/paste/gz9l2Em78lrU-jNzO9ZMLQ Here is the true openshift log: https://paste.fedoraproject.org/paste/8~5kqzziqSIdA0GQMBZqvA

[scoheb]

@gabemontero

[gabemontero]

hey @scoheb - sorry, I'm not following the two pastebins; neither of them look like pipeline console logs; nor do I see any evidence of a startBuild call. The simply look like the output from a docker build.

I did just install 0.9.7 into a jenkins instance, then created this pipeline, where I have a source strategy bc named frontend already imported:

openshift.withCluster() {
    def frontendBC = openshift.selector("bc", "frontend")
    frontendBC.startBuild('--follow')
}

And it worked fine. Here is the first few lines of the build logs which were dumped in real time as the build progressed:

[Pipeline] _OcAction
[logs:buildconfig/frontend] build/frontend-1
[logs:buildconfig/frontend] Cloning "https://github.com/openshift/nodejs-ex.git" ...
[logs:buildconfig/frontend]     Commit: 44abae3f815a26a74d3e2585ec535f4aab74a205 (Merge pull request #136 from mordicool/patch-2)
[logs:buildconfig/frontend]     Author: Ben Parees <bparees@users.noreply.github.com>
[logs:buildconfig/frontend]     Date:   Mon Sep 4 21:45:45 2017 -0400
[logs:buildconfig/frontend] ---> Installing application source ...
[logs:buildconfig/frontend] ---> Building your Node application from source
[logs:buildconfig/frontend] npm WARN deprecated to-iso-string@0.0.2: to-iso-string has been deprecated, use @segment/to-iso-string instead.
[logs:buildconfig/frontend] npm WARN deprecated jade@0.26.3: Jade has been renamed to pug, please install the latest version of pug instead of jade

And then it ended as such:

[logs:buildconfig/frontend] Pushing image 172.30.1.1:5000/myproject/origin-nodejs-sample:latest ...
[logs:buildconfig/frontend] Pushed 0/10 layers, 1% complete
[logs:buildconfig/frontend] Pushed 1/10 layers, 18% complete
[logs:buildconfig/frontend] Pushed 2/10 layers, 30% complete
[logs:buildconfig/frontend] Pushed 3/10 layers, 39% complete
[logs:buildconfig/frontend] Pushed 4/10 layers, 47% complete
[logs:buildconfig/frontend] Pushed 5/10 layers, 56% complete
[logs:buildconfig/frontend] Pushed 6/10 layers, 65% complete
[logs:buildconfig/frontend] Pushed 7/10 layers, 73% complete
[logs:buildconfig/frontend] Pushed 8/10 layers, 84% complete
[logs:buildconfig/frontend] Pushed 9/10 layers, 100% complete
[logs:buildconfig/frontend] Pushed 10/10 layers, 100% complete
[logs:buildconfig/frontend] Push successful
[logs:buildconfig/frontend] 
[Pipeline] }
[Pipeline] // node
[Pipeline] End of Pipeline
Finished: SUCCESS

[gabemontero]

sorry, hit send too soon ... can you elaborate further on your pipeline script, how it is invoked, etc.

[scoheb]

Hi, The first pastebin is the Jenkins pipeline build console log that only includes the output from the startBuild call. The second one is the complete output from the buildconfig in Openshift.

Here is how I call startBuild:

https://github.com/scoheb/declarative-ci/blob/master/Jenkinsfile#L50

Note that I removed the "--follow" here

I presume there is something in the output that is being rejected...

[gabemontero]

Ah - ok :-) ... yeah, makes sense now.

And yeah, your theory seem plausible. The question is what exactly is rejecting the data. There are a number of layers in the *Stream related code. Certainly the "unexpected stream type" is not a message coming from this plugin's code.

@jupierce - when you originally implemented the streamStdOutToConsolePrefix stuff, did anything like what is described in https://github.com/openshift/jenkins-client-plugin/issues/55#issuecomment-329490578 occur? Or any other thoughts beyond what @scoheb and I have speculated as to what might be going on?

Otherwise, for me to zero in, I'd need a reproducible test case I could run, where I think start dropping in debug so we can narrow down where exactly things are breaking down, what odd-ball data is causing the hiccup, etc.

[scoheb]

I can provide my repo and build config...let me know.

[jupierce]

@gabemontero @scoheb Looks like it may not be plugin related by in the client itself: https://github.com/kubernetes/kubernetes/issues/47800

[gabemontero]

Ah - good catch/call @jupierce !! I was totally focused on Jenkins layer stuff and didn't connect those dots. Doubly bad, I've encountered / dealt with some of the k8s/docker logger/fs type hiccups before.

Thanks again :-)

In any event, that undoubtedly is what is going here @scoheb. Bottom line, nothing we can do about it at the plugin level at this time.

The process to pulling in the recently merged k8s fix will be:

• an upcoming openshift/origin rebase will pick up the fix (my guess is it is too late for 3.7)
• once a release version of openshift with the change arises (again, most likely 3.8), we'll update the oc binary in the jenkins image

Or you can monitor the rebase activity, and when a pre-release version is available, pull it into your scenario

[scoheb]

@jupierce @gabemontero

Thanks for looking into it! Turns out my Openshift cluster was upgraded today to 3.6 and now "--follow" works!

[gabemontero]

Cool :-) ! ... a bit surprised that k8s PR got into 3.6 given the dates, but going to climb out the rabbit hole at this point

[GolanTrevize10]

Thanks Gabe, I tried new version and it prints the logs now

[gabemontero]

thanks for the confirm @GolanTrevize10

[svenstaro]

I would like to reopen this and state that openshift.startBuild('stuff', '--follow') doesn't actually work and doesn't display the log in real time while building. This is openshift 3.10. However, using selector.logs() does work.

[gabemontero]

@svenstaro I've reproduced and submitted https://github.com/openshift/jenkins-client-plugin/pull/189 to address

The cause was unrelated to what was worked last year, but rather some fallout from a refactor off of durable task plugin we had to take on for other reasons.

So not going to reopen this per se ... just tracking with the PR