j3t / jenkins-pipeline-cache-plugin

A cloud native file cache for Jenkins build pipelines which uses an S3-Bucket as storage provider.
MIT License
15 stars 2 forks source link

how about a feature to support run block if no hit the cache. #14

Open wuhuizuo opened 2 years ago

wuhuizuo commented 2 years ago

New feature

The block will not run. if key exist:

cache(key: "cache-key-balabala", onlyRunWhenMiss: true, ...) {
  //  steps
}
wuhuizuo commented 2 years ago

I'm not familiar with java. @j3t

j3t commented 2 years ago

Do you have some examples where this could be useful? For git, you could just use shallow clones instead.

wuhuizuo commented 2 years ago

I have the example, most is to save binary building time.

  1. found in cache binary, if not found the cache then build in closure block.
  2. some binary build step are same cross pipelines, but reusing a slave pipeline will make many simple pipeline to create.
  3. fetching from s3 is faster than fetching from Jenkins artifacts.
j3t commented 2 years ago

I have the example, most is to save binary building time.

I get what you want and I guess we can make it happen but for me the use case is still not clear. Do you have a concrete example?

wuhuizuo commented 2 years ago

I have the example, most is to save binary building time.

I get what you want and I guess we can make it happen but for me the use case is still not clear. Do you have a concrete example?

In test pipeline, design steps:

stage("prepare binary") {
  steps {
      cache(path: "bin/dir1", key:"uniq-key", onlyRunWhenMiss: true) {
         sh "./build.sh" // it take several minutes or hours
       }
      cache(path: "bin/dir2", key:"uniq-key", onlyRunWhenMiss: true) {
         sh "./build2.sh" // it take several minutes or hours
       }
   }    
}

stage("test") {
  steps {
      sh "./run-test.sh" // using bin/dir1 and /bin/dir2
   }    
}
j3t commented 2 years ago

I'm not sure, I mean you could just add a condition like

...
      cache(path: "bin/dir1", key:"uniq-key") {
         sh "[ ! -d "bin/dir1" ] && ./build.sh"
      }
...

or you could extend the build.sh so that the resources are not regenerated if they already exist?

Also, what is about restore keys? They are ignored in this context, right?