eddiewebb / circleci-queue

CircleCI orb to block/queue jobs to enforce max concurrency limits
MIT License
74 stars 75 forks source link

Change in CircleCI API behaviour seems to have broken .build_parameters.CIRCLE_JOB check #70

Closed microbit-matt-hillsdon closed 3 years ago

microbit-matt-hillsdon commented 3 years ago

Thanks for a really useful Orb!

Orb version

1.6.1

What happened

main queueable
This build will block until all previous builds complete.
Max Queue Time: 60 minutes.
Only blocking execution if running previous jobs on branch: main
Attempting to access CircleCI api. If the build process fails after this step, ensure your CIRCLECI_API_KEY is set.
API access successful
Checking time of workflow: c64f3cde-8b5f-4b97-bf45-9065d7f38658
Workflow was created at: 2021-10-21T08:05:14Z
Checking time of workflow: 07ab1c33-cfbf-4ca4-81f4-81a96177c600
Workflow was created at: 2021-10-21T08:04:43Z
Orb parameter block-workflow is false.
Only blocking execution if running previous jobs matching this job: staging
Oldest job: null
This Workflow Timestamp: "2021-10-21T08:04:43Z"
Oldest Workflow Timestamp: null
API shows no previous jobs/workflows, but it is possible a previous workflow has pending jobs not yet visible in API.
Rerunning check 1/1
Only blocking execution if running previous jobs on branch: main
Attempting to access CircleCI api. If the build process fails after this step, ensure your CIRCLECI_API_KEY is set.
API access successful
Checking time of workflow: c64f3cde-8b5f-4b97-bf45-9065d7f38658
Workflow was created at: 2021-10-21T08:05:14Z
Checking time of workflow: 07ab1c33-cfbf-4ca4-81f4-81a96177c600
Workflow was created at: 2021-10-21T08:04:43Z
Orb parameter block-workflow is false.
Only blocking execution if running previous jobs matching this job: staging
Oldest job: null
This Workflow Timestamp: "2021-10-21T08:04:43Z"
Oldest Workflow Timestamp: null
Front of the line, WooHoo!, Build continuing
CircleCI received exit code 0

As you can see from the last few lines, the oldest job and oldest workflow aren't being correctly calculated. From debugging the jq, the cause is the build_parameters field in the JSON which is an empty object. I assume it used to contain the CIRCLE_ environment variables but I'm not familiar with this API.

Perhaps this is a CircleCI API change?

It may be possible to use .workflows.job_name instead.

I can't find API documentation that covers either field (or even the /tree/ endpoint).

I discovered this issue with an older version of the orb, 1.0.110. In that version the null return causes the step to fail, which was fortunate really or else I would not have noticed. It might be good to introduce a sanity check in the latest version.

Expected behavior

It should correctly calculate the oldest job and block as needed.

Workarounds

Set block-workflow and live with the reduced concurrency.

eirikur-grid commented 3 years ago

Running into this as well. We use this orb (version 1.5.0) to serialize deployments but recently we've started seeing deployment conflicts. Here's the log output for a deployment job that should wait but does not.

This build will block until all previous builds complete.
Max Queue Time: 20 minutes.
Only blocking execution if running previous jobs on branch: develop
Attempting to access CircleCI api. If the build process fails after this step, ensure your CIRCLECI_API_KEY is set.
API access successful
Checking time of workflow: 1e0670d6-d69c-4346-80dd-45588c614a28
Workflow was created at: 2021-10-28T10:34:57Z
Checking time of workflow: dc9d7020-b642-4b10-848f-c5f4bcf1f017
Workflow was created at: 2021-10-28T10:34:35Z
Orb parameter block-workflow is false.
Only blocking execution if running previous jobs matching this job: deploy_dev_foxtrot
Oldest job: null
This Workflow Timestamp: "2021-10-28T10:34:57Z"
Oldest Workflow Timestamp: null
API shows no previous jobs/workflows, but it is possible a previous workflow has pending jobs not yet visible in API.
Rerunning check 1/1
Only blocking execution if running previous jobs on branch: develop
Attempting to access CircleCI api. If the build process fails after this step, ensure your CIRCLECI_API_KEY is set.
API access successful
Checking time of workflow: 1e0670d6-d69c-4346-80dd-45588c614a28
Workflow was created at: 2021-10-28T10:34:57Z
Checking time of workflow: dc9d7020-b642-4b10-848f-c5f4bcf1f017
Workflow was created at: 2021-10-28T10:34:35Z
Orb parameter block-workflow is false.
Only blocking execution if running previous jobs matching this job: deploy_dev_foxtrot
Oldest job: null
This Workflow Timestamp: "2021-10-28T10:34:57Z"
Oldest Workflow Timestamp: null
Front of the line, WooHoo!, Build continuing

CircleCI received exit code 0
eddiewebb commented 3 years ago

Looks like API no longer returns JOB_NAME

 "build_parameters":{
    "CIRCLE_JOB":"singlejob"
  },

Is now always

 "build_parameters":{
  },

but workflows.job-name has it, so should fix.

eddiewebb commented 3 years ago

Thanks @microbit-matt-hillsdon , and sorry for delay.

I have pushed change on #71 that should fix this, passing locally. In a about 10 minutes it should spit out a dev version to try, if anyone gets a chance to try before tomorrow let me know, otherwise Ill run E2E tomorrow.

eddiewebb commented 3 years ago

@microbit-matt-hillsdon , @eirikur-grid, @cvs-sigrejas, @bhb603, @jakemhiller - this is now fixed in 1.6.2

eddiewebb/queue@1.6.2

microbit-matt-hillsdon commented 3 years ago

I've upgraded and triggered some builds to check the blocking behaviour and all seems to be working well for me. Thanks for the fix @eddiewebb 🎉