CI/CD checks failing preventing scheduled new Contributor Spotlight from being published

jenkins-infra / helpdesk

Open your Infrastructure related issues here for the Jenkins project

https://github.com/jenkins-infra/helpdesk/issues/new/choose

16 stars 10 forks source link

CI/CD checks failing preventing scheduled new Contributor Spotlight from being published #4284

Closed krisstern closed 2 weeks ago

krisstern commented 2 weeks ago

Service(s)

infra.ci.jenkins.io, contributors.jenkins.io

Summary

It appears some infra issue is preventing the Contributor Spotlight https://github.com/jenkins-infra/contributor-spotlight/pull/195 from being published today. The error appears to be as follows:

# Deploy PR to preview site / Shell Script

Error in sh step, with arguments netlify-deploy --draft=true --siteName "contributor-spotlight" --title "Preview deploy for ${CHANGE_ID}" --alias "deploy-preview-${CHANGE_ID}" -d ./public.

Could someone from the Infra team please take a look and see if the issue can be resolved today EDT? Thanks!

Reproduction steps

No response

krisstern commented 2 weeks ago

c.c. @kmartens27 @gounthar @alyssat

dduportal commented 2 weeks ago

Hi @krisstern , thanks for raising this issue!

Sorry for the late communication, but all website previews (e.g. the checks running on infra.ci) are disabled since sunday as per https://github.com/jenkins-infra/helpdesk/issues/4282.

We'll enable them as soon as possible, but no immediate action are possible. Sorry for the inconvenience: we kept it private as it could have been a serious security matter.

krisstern commented 2 weeks ago

Thanks @dduportal for the info!

dduportal commented 2 weeks ago

Update:

The error is related to permission with the token being used. I might have messed something when rotating it. Currently checking
Once fixed, all maintainers or admin of any website (contributors spotlights, jenkins.io,e tc.) should be able to have preview sites for their PRs. Only non contributors should not have it.

dduportal commented 2 weeks ago

Update:

Cleaned-up the Netlify organization:
- Enforced 2FA (including for the technical user jenkins-infra-team, with the shared totp in our encrypted private secret repository)
- Removed all tokens for jenkins-infra-team user and created a new one valid 90 days only
- Added the token in infra.ci (manually AND in config as code - sops)
- Changed the permissions to "Developer" for this technical user so it can only write (e.g. deploy)

=> PR https://github.com/jenkins-infra/contributor-spotlight/pull/195 build successfully deployed the preview site. We'll watch the production deployment next.

gounthar commented 2 weeks ago

Thank you so much for solving that. 👍

dduportal commented 2 weeks ago

I'm closing the issue as the problem seems gone. Feel free to reopen it with details if it happens again!

krisstern commented 2 weeks ago

Thanks for resolving this issue for us in such a prompt manner @dduportal!