projectglow / glow

An open-source toolkit for large-scale genomic analysis
https://projectglow.io
Apache License 2.0
262 stars 107 forks source link

Optimize CircleCI Config #505

Closed dvcastillo closed 2 years ago

dvcastillo commented 2 years ago

Signed-off-by: David Castillo d.v.castillo37@gmail.com

What changes are proposed in this pull request?

Update the following orbs:

  1. codecov/codecov@1.0.5 ==> codecov/codecov@3.2.2
  2. circleci/jq@1.6.0 ==> circleci/jq@2.2.0

Based on looking at the resource utilization of all CircleCI workflows, I made the following optimizations for credit consumption:

  1. For the job, check-docs, the overall resource utilization for both RAM/CPU is much lower than this resource class is intended for (screenshots included). I have added an additional resource class size of medium+ to minimize credit consumption.
  2. For the job, all-notebook-tests, the overall resource utilization for both RAM/CPU is much lower than this resource class is intended for (screenshots included). I have added an additional resource class size of medium+ to minimize credit consumption. Screen Shot 2022-04-01 at 10 37 43 AM Screen Shot 2022-04-01 at 10 31 27 AM

How is this patch tested?

(Details)

codecov[bot] commented 2 years ago

Codecov Report

Merging #505 (e991daa) into master (0eb8767) will not change coverage. The diff coverage is n/a.

@@           Coverage Diff           @@
##           master     #505   +/-   ##
=======================================
  Coverage   92.53%   92.53%           
=======================================
  Files          95       95           
  Lines        4941     4941           
  Branches      478      478           
=======================================
  Hits         4572     4572           
  Misses        369      369           

Continue to review full report at Codecov.

Legend - Click here to learn more Δ = absolute <relative> (impact), ø = not affected, ? = missing data Powered by Codecov. Last update 0eb8767...e991daa. Read the comment docs.

dvcastillo commented 2 years ago

@williambrandler Please review when you get a chance.

williambrandler commented 2 years ago

aha, yes that's right, check-docs and all-notebook-tests do not use much resources

I updated to x-large for the scala + spark tests, but did not know how to set conditions for the other circleci tests, so applied it to all tests. Thanks for adding this