search

jenkins-x / jx

Jenkins X provides automated CI+CD for Kubernetes with Preview Environments on Pull Requests using Cloud Native pipelines from Tekton
https://jenkins-x.io/
Apache License 2.0
4.56k stars 786 forks source link

Infinite Loop: Failed to query the Pull Request last commit status #6532

Closed vlindvall closed 4 years ago

vlindvall commented 4 years ago

Summary

When promoting to the staging environment, I typically will see a couple warning messages when running jx promote -b --all auto --timeout 1h --version $(cat ../ ../VERSION) saying:

WARNING: Failed to query the Pull Request last commit status for https://github.com/mygit/my-environment-staging/pull/number ref XYZ Could not find a status for repository mygit/my-environment-staging with ref XYZ

But will eventually get git provider status pending from PR. And merge status checks all pass and the promotion worked. This has happened multiple times on the same project.

However, I am now experiencing an infinite loop when promoting to stage with the same warning message as above. It will continue trying until an hour passes, and subsequently fails.

The only change I had made between a past successful build and this failing infinite loop build was adding a couple environment variables in the values.yaml file

Steps to reproduce the behavior

Build Now in Jenkins console or make changes in jx namespace, commit/push those changes to git and build happens automatically

Expected behavior

Successful build to staging environment

Actual behavior

Infinite Warning Loop

Jx version

2.0.1084

The output of jx version is:

NAME               VERSION
jx                 2.0.1084
jenkins x platform 2.0.1787
Kubernetes cluster v1.14.9-eks-c0eccc
kubectl            v1.17.0
helm client        Client: v2.14.3+g0e7f3b6
git                2.17.1
Operating System   Ubuntu 18.04.3 LTS

Jenkins type

Kubernetes cluster

jx create cluster eks

Operating system / Environment

Ubuntu 18.04.3 LTS

vlindvall commented 4 years ago

Solved by adding --no-wait=true --no-poll=true to my jx promote command

srehmanproov commented 4 years ago

I am seeing this with auto promotion to staging as well, is there any fix for this or do we need whitelist any IPs for gitHub to work with GKE?

vlindvall commented 4 years ago

I am seeing this with auto promotion to staging as well, is there any fix for this or do we need whitelist any IPs for gitHub to work with GKE?

I posted my solution above which fixed the problem for me, but made it so that I had to manually merge in staging repo.

Ended up just creating a new cluster with jenkins x and starting over. Haven't run into the problem again yet

srehmanproov commented 4 years ago

Issue I see is even after PR is merged to staging repo, the staging pipeline does not get triggered. All PRs to staging are auto merged.

jenkins-x-bot commented 4 years ago

Issues go stale after 90d of inactivity. Mark the issue as fresh with /remove-lifecycle stale. Stale issues rot after an additional 30d of inactivity and eventually close. If this issue is safe to close now please do so with /close. Provide feedback via https://jenkins-x.io/community. /lifecycle stale

mmugabo-rockhs commented 4 years ago

Getting the same issue: "Failed to query the Pull Request last commit status for...." when I create an app with "jx create quickstart" and during promote to staging env, that is when this error happens.

This is my environment:

NAME VERSION jx 2.1.46+cjxd.10 Kubernetes cluster v1.14.10-gke.36 kubectl v1.13.2 helm client 2.12.2 git 2.21.0 Operating System Mac OS X 10.14.6 build 18G95

I have created my set up using the following:

Please let me know how ot move on this.. Thanks

covid82 commented 4 years ago

Getting the same issue: "Failed to query the Pull Request last commit status for...." when I create an app with "jx create quickstart" and during promote to staging env, that is when this error happens.

This is my environment:

NAME VERSION jx 2.1.46+cjxd.10 Kubernetes cluster v1.14.10-gke.36 kubectl v1.13.2 helm client 2.12.2 git 2.21.0 Operating System Mac OS X 10.14.6 build 18G95

I have created my set up using the following:

  • Created my GKE cluster thru the console
  • Forked the Cloudbees jenkins-x-config set up repo
  • I have TLS enabled with my domain.
  • I have everything else set up such as Github Org, bot account, etc...

Please let me know how ot move on this.. Thanks

I had a similar problem on AWS. Promotion worked well for a couple of builds, but after that started to go into Failed to query the Pull Request last commit status loop. I noticed that jx was unable to create maven pod due to insufficient CPU. I recreated a cluster with more powerful nodes (m2.xlarge in my case) which seems to solved the problem.

developer-rockhs commented 4 years ago

@covid82 Thanks for your response.

I went back to the most basic jx boot config and then I did a step by step adding advanced configurations one at a time (doing a fresh install every time) to be able to single out the issue. And that led to me to my TLS, the whole CI/CD works perfectly if I do not enable TLS. When I have it enabled, I can't get an application promoted to staging (after jx create quickstart), I get into an infinite loop with this error:

got git provider status pending from PR https://github.com/rockhs-org/environment-core-cluster-staging/pull/1

Do you know what logs to look into in this scenario? By enabling TLS I mean b4 running jx boot, in my jx-requirement.yaml I set the following ingress props:

ingress:
  domain: core.MYDOMAIN.com
  externalDNS: true
  namespaceSubDomain: -jx.
  tls:
    email: EMAIL@MYDOMAIN.com
    enabled: true
    production: true

Thanks.

jenkins-x-bot commented 4 years ago

Stale issues rot after 30d of inactivity. Mark the issue as fresh with /remove-lifecycle rotten. Rotten issues close after an additional 30d of inactivity. If this issue is safe to close now please do so with /close. Provide feedback via https://jenkins-x.io/community. /lifecycle rotten

jenkins-x-bot commented 4 years ago

Rotten issues close after 30d of inactivity. Reopen the issue with /reopen. Mark the issue as fresh with /remove-lifecycle rotten. Provide feedback via https://jenkins-x.io/community. /close

jenkins-x-bot commented 4 years ago

@jenkins-x-bot: Closing this issue.

In response to [this](https://github.com/jenkins-x/jx/issues/6532#issuecomment-670089318): >Rotten issues close after 30d of inactivity. >Reopen the issue with `/reopen`. >Mark the issue as fresh with `/remove-lifecycle rotten`. >Provide feedback via https://jenkins-x.io/community. >/close Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md). If you have questions or suggestions related to my behavior, please file an issue against the [jenkins-x/lighthouse](https://github.com/jenkins-x/lighthouse/issues/new?title=Command%20issue:) repository.