jenkinsci / jobcacher-plugin

Jenkins plugin that improves build performance for transient agents by caching files
https://plugins.jenkins.io/jobcacher/
MIT License
31 stars 32 forks source link

Cache is recreated when hit on default branch #335

Open dragenet opened 2 months ago

dragenet commented 2 months ago

Jenkins and plugins versions report

Environment ```text Jenkins: 2.462.1 OS: Linux - 5.15.0-117-generic Java: 21.0.4 - Eclipse Adoptium (OpenJDK 64-Bit Server VM) --- active-directory:2.36 antisamy-markup-formatter:162.v0e6ec0fcfcf6 apache-httpcomponents-client-4-api:4.5.14-208.v438351942757 asm-api:9.7-33.v4d23ef79fcc8 atlassian-bitbucket-server-integration:4.0.0 authentication-tokens:1.119.v50285141b_7e1 blueocean:1.27.14 blueocean-bitbucket-pipeline:1.27.14 blueocean-commons:1.27.14 blueocean-config:1.27.14 blueocean-core-js:1.27.14 blueocean-dashboard:1.27.14 blueocean-display-url:2.4.3 blueocean-events:1.27.14 blueocean-git-pipeline:1.27.14 blueocean-github-pipeline:1.27.14 blueocean-i18n:1.27.14 blueocean-jwt:1.27.14 blueocean-personalization:1.27.14 blueocean-pipeline-api-impl:1.27.14 blueocean-pipeline-editor:1.27.14 blueocean-pipeline-scm-api:1.27.14 blueocean-rest:1.27.14 blueocean-rest-impl:1.27.14 blueocean-web:1.27.14 bootstrap5-api:5.3.3-1 bouncycastle-api:2.30.1.78.1-248.ve27176eb_46cb_ branch-api:2.1178.v969d9eb_c728e caffeine-api:3.1.8-133.v17b_1ff2e0599 checks-api:2.2.0 cloudbees-bitbucket-branch-source:888.v8e6d479a_1730 cloudbees-folder:6.942.vb_43318a_156b_2 command-launcher:115.vd8b_301cc15d0 commons-lang3-api:3.14.0-76.vda_5591261cfe commons-text-api:1.12.0-129.v99a_50df237f7 configuration-as-code:1836.vccda_4a_122a_a_e credentials:1371.vfee6b_095f0a_3 credentials-binding:681.vf91669a_32e45 display-url-api:2.204.vf6fddd8a_8b_e9 durable-task:568.v8fb_5c57e8417 echarts-api:5.5.0-1 eddsa-api:0.3.0-4.v84c6f0f4969e favorite:2.221.v19ca_666b_62f5 folder-properties:1.2.1 font-awesome-api:6.5.2-1 git:5.3.0 git-client:5.0.0 github:1.40.0 github-api:1.321-468.v6a_9f5f2d5a_7e github-branch-source:1793.v1831e9c68d77 gson-api:2.11.0-41.v019fcf6125dc handy-uri-templates-2-api:2.1.8-30.v7e777411b_148 hashicorp-vault-plugin:368.v48134f694db_f htmlpublisher:1.36 instance-identity:185.v303dc7c645f9 ionicons-api:74.v93d5eb_813d5f jackson2-api:2.17.0-379.v02de8ec9f64c jakarta-activation-api:2.1.3-1 jakarta-mail-api:2.1.3-1 javadoc:280.v050b_5c849f69 javax-activation-api:1.2.0-7 javax-mail-api:1.6.2-10 jaxb:2.3.9-1 jdk-tool:80.v8a_dee33ed6f0 jenkins-design-language:1.27.14 jjwt-api:0.11.5-112.ve82dfb_224b_a_d jobcacher:481.v15f51ca_4c6b_7 joda-time-api:2.12.7-29.v5a_b_e3a_82269a_ jquery:1.12.4-1 jquery3-api:3.7.1-2 jsch:0.2.16-86.v42e010d9484b_ json-api:20240303-41.v94e11e6de726 json-path-api:2.9.0-58.v62e3e85b_a_655 junit:1284.vf75d778f98c5 kubernetes:4280.vd919fa_528c7e kubernetes-client-api:6.10.0-240.v57880ce8b_0b_2 kubernetes-credentials:174.va_36e093562d9 mailer:472.vf7c289a_4b_420 matrix-auth:3.2.2 matrix-project:832.va_66e270d2946 maven-plugin:3.23 metrics:4.2.21-451.vd51df8df52ec mina-sshd-api-common:2.13.1-117.v2f1a_b_66ff91d mina-sshd-api-core:2.13.1-117.v2f1a_b_66ff91d okhttp-api:4.11.0-172.vda_da_1feeb_c6e pipeline-build-step:540.vb_e8849e1a_b_d8 pipeline-graph-analysis:216.vfd8b_ece330ca_ pipeline-groovy-lib:727.ve832a_9244dfa_ pipeline-input-step:495.ve9c153f6067b_ pipeline-milestone-step:119.vdfdc43fc3b_9a_ pipeline-model-api:2.2205.vc9522a_9d5711 pipeline-model-definition:2.2205.vc9522a_9d5711 pipeline-model-extensions:2.2205.vc9522a_9d5711 pipeline-rest-api:2.34 pipeline-stage-step:312.v8cd10304c27a_ pipeline-stage-tags-metadata:2.2205.vc9522a_9d5711 pipeline-stage-view:2.34 plain-credentials:183.va_de8f1dd5a_2b_ plugin-util-api:4.1.0 prism-api:1.29.0-15 prometheus:780.v7c50a_d288424 pubsub-light:1.18 scm-api:696.v778d637b_a_762 script-security:1341.va_2819b_414686 sectioned-view:1.27 slack:734.v7f9ec8b_66975 snakeyaml-api:2.2-121.v5a_68b_9300b_d4 sse-gateway:1.27 ssh-credentials:343.v884f71d78167 sshd:3.330.vc866a_8389b_58 structs:338.v848422169819 token-macro:400.v35420b_922dcb_ trilead-api:2.147.vb_73cc728a_32e variant:60.v7290fc0eb_b_cd workflow-aggregator:600.vb_57cdd26fdd7 workflow-api:1322.v857eeeea_9902 workflow-basic-steps:1058.vcb_fc1e3a_21a_9 workflow-cps:3922.va_f73b_7c4246b_ workflow-durable-task-step:1364.v2fd76fb_6fd41 workflow-job:1436.vfa_244484591f workflow-multibranch:795.ve0cb_1f45ca_9a_ workflow-scm-step:427.v4ca_6512e7df1 workflow-step-api:678.v3ee58b_469476 workflow-support:920.v59f71ce16f04 xray-connector:2.6.1 ```

What Operating System are you using (both controller, and any agents involved in the problem)?

I'm using Jenkins on Kubernetes:

Reproduction steps

  1. Run job on master branch with cached yarn install with the following cache config cache(maxCacheSize: 10000, defaultBranch: 'master', caches:[ arbitraryFileCache( cacheName: 'yarn_install_cache', path: './', includes: '**/node_modules/**/*,.yarn/install-state.gz', excludes: '**/node_modules/@repo/**/*', cacheValidityDecidingFile: 'yarn.lock', compressionMethod: 'TAR_ZSTD' ) ]) { sh 'yarn install' }
  2. Run job on another branch with cached yarn install with the same cache config

Expected Results

I think when we have cache hit from default caches it should be copied to job caches, not created again from scratch

Actual Results

` [Cache for ./ (yarn_install_cache) with id 0374ec857381230fa83049643ce3cdee] Searching cache in job specific caches... [Cache for ./ (yarn_install_cache) with id 0374ec857381230fa83049643ce3cdee] Searching cache in default caches... [Cache for ./ (yarn_install_cache) with id 0374ec857381230fa83049643ce3cdee] got hash 1bb160d6d3bd445af936d09bbc7acc99 for cacheValidityDecidingFile(s) - actual file(s): /home/jenkins/agent/workspace/ext-mui-turborepo-template_PR-45/yarn.lock [Cache for ./ (yarn_install_cache) with id 0374ec857381230fa83049643ce3cdee] Found cache in default caches [Cache for ./ (yarn_install_cache) with id 0374ec857381230fa83049643ce3cdee] Restoring cache... [Cache for ./ (yarn_install_cache) with id 0374ec857381230fa83049643ce3cdee] Cache restored in 52182ms [Pipeline] { [Pipeline] sh

Anything else?

I checked it also with yarn install --immutable to make sure it doesn't modify yarn.lock and I have the same result

Are you interested in contributing a fix?

No response

repolevedavaj commented 2 months ago

Thanks for your suggestion. I agree, maybe it should not even be copied when the default cache is valid. Especially when cache creation needs so much time...