eddiewebb / circleci-queue

CircleCI orb to block/queue jobs to enforce max concurrency limits
MIT License
74 stars 75 forks source link

Fix: blocking is based on workflow start date instead of committer date [semver:minor] #37

Closed kobim closed 5 years ago

kobim commented 5 years ago

CircleCI v2 API now contains workflow information, and its created_at might be a better candidate for comparison than committer_date.

Fixes #22

eddiewebb commented 5 years ago

Excellent. I was thinking about the same but going up to pipeline level to get the 'received at' information for the pipeline trigger. But this should be close enough and skips an extra call. Will review this in more detail soon

eddiewebb commented 5 years ago

@kobim looks like tests are failing. I dont expect everyone to pick up BATS, so I can take a look.

(if you;re interested..)

brew install bats
...
bats test/

EDIT: interesting side effect of running tests in CCI environment:

Queueing on forks is not supported. Skipping queue...

I've explicitly null'd those values for testing.

eddiewebb commented 5 years ago

Dev version of orb available for testing - eddiewebb/queue@dev:pull/37-32498ab0ef313e9f47c48e76a7de514b454ed881

mu-bro commented 5 years ago

I always have a queue job failed on the second loop. Scenario to reproduce: To start 2 workflows. In the first workflow queue/block_workflow doesn't block the pipeline - it is correct. And in second workflow 'queue/block_workflow' should wait first one to be finished. First loop iteration goes fine, but failed on the second loop iteration. Logs:

my-branch-name queueable This build will block until all previous builds complete. Max Queue Time: 60 minutes. Only blocking execution if running previous jobs on branch: my-branch-name Attempting to access CircleCI api. If the build process fails after this step, ensure your CIRCLECI_API_KEY is set. API access successful Orb parameter block-workflow is true. This job will block until no previous workflows have any jobs running. Oldest job: 108991 This Workflow Timestamp: "2019-10-18T09:28:59Z" Oldest Workflow Timestamp: "2019-10-18T09:26:24Z" This build (108997) is queued, waiting for build number (108991) to complete. Total Queue time: 0 seconds. Only blocking execution if running previous jobs on branch: my-branch-name Attempting to access CircleCI api. If the build process fails after this step, ensure your CIRCLECI_API_KEY is set. API access successful Orb parameter block-workflow is true. This job will block until no previous workflows have any jobs running. Oldest job: API Call for existing jobs failed, failing this build. Please check API token

For some reason /tmp/jobstatus.json become empty even there are no problems with API requests. Unfortunately, I cannot tell where it fails exactly, as you suppress the error in curl -f -s $jobs_api_url_template > /tmp/jobstatus.json and echo "API access successful" will be printed in any scenario. I can just assume that on this line under some circumstances it writes empty result in '/tmp/jobstatus.json' cat /tmp/jobstatus.json | jq --arg created_at $created_at --arg workflow ${workflow} '(.[] | select(.workflows.workflow_id == $workflow) | .workflows) |= . + {created_at:$created_at}' | tee /tmp/jobstatus.json

eddiewebb commented 5 years ago

@mu-bro - additional API keys are not needed. The same user level API tokens can be used for V1 and v2.

I'll look at the other issue you raised.

eddiewebb commented 5 years ago

bors try

mu-bro commented 5 years ago

@eddiewebb about additional API - yes, you right. That's why I removed my comment about it

eddiewebb commented 5 years ago

bors try

bors[bot] commented 5 years ago

try

Already running a review

eddiewebb commented 5 years ago

bors try-

eddiewebb commented 5 years ago

bors try

eddiewebb commented 5 years ago

There is something afoul with the jq command that augments job detauls with timestamp of workflow causing the file to be written empty.

  madethisup queueable
This build will block until all previous builds complete. 
Max Queue Time: 1 minutes. 
Only blocking execution if running previous jobs on branch: madethisup 
Using test mock response 
Checking time of workflow: ec6172d3-b286-4d3a-a1da-c4efba0a0c05 
Using test mock workflow response 
Workflow was created at: 2019-01-17T11:17:20-05:00 
new augmented_jobstatus.json:   #empty file...
Orb parameter block-workflow is false
bors[bot] commented 5 years ago

try

Timed out

eddiewebb commented 5 years ago

It seems from local testing the issue was that jq was writing an empty file before fully processing it as input. I modiefied script to use temporary intermediate files to avoid.

eddiewebb commented 5 years ago

bors try

eddiewebb commented 5 years ago

BotComment: Development version of orb available for manual validation - eddiewebb/queue@dev:37

eddiewebb commented 5 years ago

@mu-bro - can you give eddiewebb/queue@dev:37 a shot?

bors[bot] commented 5 years ago

try

Timed out

eddiewebb commented 5 years ago

bors try (i've switched CCI to use checks and bors.toml to require all jobs)

eddiewebb commented 5 years ago

BotComment: Development version of orb available for manual validation - eddiewebb/queue@dev:37

eddiewebb commented 5 years ago

bors try-

mu-bro commented 5 years ago

@eddiewebb , confirm that it works correct for my tests.

eddiewebb commented 5 years ago

bors merge

eddiewebb commented 5 years ago

BotComment: Development version of orb available for manual validation - eddiewebb/queue@dev:37

eddiewebb commented 5 years ago

BotComment: Production version of orb available for use -

bors[bot] commented 5 years ago

Build succeeded