timja / jenkins-gh-issues-poc-06-18

0 stars 0 forks source link

[JENKINS-67903] Jenkins - "Build after other projects are built" not working for pipeline jobs #1857

Open timja opened 2 years ago

timja commented 2 years ago

I have a K8s Jenkins server in which I'm trying to trigger job Y after job X finished running.

I've added a definition in job_y as follows:

While this is working just fine for "freestyle job", nothing happens on those 2 "pipeline jobs"

To sum up:

freestyle_job_x ==> freestyle_job_y (working)

pipeline_job_x ==> pipeline_job_y (not working)

I do want to mention that the exact same experiment worked just fine in "on prem" Jenkins server (i.e. also pipeline jobs were working)

Why is that?

 


Originally reported by noa_zemer, imported from: Jenkins - "Build after other projects are built" not working for pipeline jobs
  • status: Open
  • priority: Critical
  • resolution: Unresolved
  • imported: 2022/01/10
timja commented 2 years ago

kon:

"Build after other projects are built" is jenkins.triggers.ReverseBuildTrigger (javadoc, config.jelly) defined in core. I don't know what causes the problem, but it's unlikely to be related to pipeline-build-step-plugin.

Which version of Jenkins is this?
Are the projects located in folders?
Are these single-branch or multibranch pipeline projects?

timja commented 2 years ago

JIRAUSER140674:

**

Hello and thanks for your quick response!

Jenkins version: Jenkins 2.322

I tried upgrading and downgrading the server, the triggering problem remains. 

Are the projects located in folders? not sure I understood the problem. Both jobs are on the same Jenkins.  
single-branch or multibranch pipeline projects:  single-branch pipeline projects.

I will also say we use configuration as code.

This Jenkins server is redeployed once a week, and it seems the deployment affects whether the Jobs are triggered or not, even when no new code is pushed.

 

timja commented 2 years ago

kon:

the deployment affects whether the Jobs are triggered or not

Do you mean that, if you redeploy the Jenkins server and build pipeline_job_x, then pipeline_job_y does not trigger; but if you later build pipeline_job_x again without redeploying Jenkins, then pipeline_job_y triggers correctly?

If so, I wonder if this could be caused by Jenkins not loading pipeline_job_y before it is explicitly referenced somehow, and thus not noticing that pipeline_job_y has a ReverseBuildTrigger. But I have not checked whether Jenkins could postpone loading in that way.

timja commented 2 years ago

kon:

When you redeploy the Jenkins server, do you use Job DSL to set up the projects?

timja commented 2 years ago

kon:

Maybe related to JENKINS-64918 as well.

timja commented 2 years ago

JIRAUSER140674:

No, something even stranger happens: the triggering sometimes works and sometimes stops working. The change always happen after the Jenkins is redeployed (once a week automatically). It means that jobs can be triggered after other jobs as they should for a whole week, and then after deployment, they stopped being triggered, and then the week after that, they go back to being triggered. Nothing seems to change on the configuration files of the jobs. Even when they are not triggered, their job configuration shows they are configured to be triggered.

I tried testing the deployment issue myself - I deployed the Jenkins a few times in a row, one time after another: First time the triggering did not work, second time it did, third time it did, and then 4th, 5th and 6th try, it did not, and so on...

I thought maybe there is a Jenkins variable behind scenes, which is connected to job triggering, and sometimes it gets loaded in the correct value, but other times it does not...

 

timja commented 2 years ago

JIRAUSER140674:

I will add that we do use Job DSL, but even when I created jobs manually, and configured job a to run after job b, the triggering does not work - these jobs behave exactly like the jobs created by Job DSL (meaning when triggering goes back to working, job a also trigger job b).

Also, I will mention that triggering freeStyle jobs does work, no matter what. Meaning freeStyle job a triggers successfully freeStyle job b.

 

 

   

timja commented 2 years ago

kon:

If you enable FINE logging for jenkins.triggers.ReverseBuildTrigger, then ReverseBuildTrigger.calculateCache() should log "from {0} see upstreams {1}" when it scans the projects. Perhaps that will reveal some difference between the deployments.