nienbo / cache-buildkite-plugin

Tarball, Rsync & S3 Cache Kit for Buildkite. Supports Linux, macOS and Windows
https://buildkite.com/plugins
MIT License
67 stars 39 forks source link

[FEATURE REQUEST] Ability to run a command before calculating cache key? #47

Open willhughes-au opened 2 years ago

willhughes-au commented 2 years ago

I have a situation where it would be useful to cache data, but the key needs to be a checksum of a file that isn't on the filesystem, yet.

An arbitrary example:

  - label: "Run Cached Step"
    command:
      - wget -O remote-config https://example.com
      - ./script-that-does-something.sh
    plugins:
      - gencer/cache#v2.4.10:
          key: "{{ id }}-{{ checksum 'remote-config' }}"
          paths:
            - .cache-dir

Running the wget command as a separate step won't work, as the file system gets cleaned, and even if there weren't, we run multiple agents.

Would it be possible/make sense to have a pre-run-command argument that I could have, Or perhaps be able to invoke the plugin manually?

Examples:

  - label: "Run Cached Step"
    command:
      - ./script-that-does-something.sh
    plugins:
      - gencer/cache#v2.4.10:
          key: "{{ id }}-{{ checksum 'remote-config' }}"
          pre-restore-command: wget -O remote-config https://example.com
          paths:
            - .cache-dir

or perhaps:

  - label: "Run Cached Step"
    command:
      - wget -O remote-config https://example.com
      - gencer-cache-restore
      - ./script-that-does-something.sh
    plugins:
      - gencer/cache#v2.4.10:
          key: "{{ id }}-{{ checksum 'remote-config' }}"
          defer_restore: true
          paths:
            - .cache-dir

This is probably a very niche/weird request.

gencer commented 2 years ago

This is probably a very niche/weird request.

Normally, Yes. But not in this situation because of:

Running the wget command as a separate step won't work, as the file system gets cleaned, and even if there weren't, we run multiple agents.

Which makes this request meaningful.

There are 3 open requests incl. this one and I still not yet finished any of them. I'll try to implement ASAP. However, I am fully open for PRs (Generally speaking).

stale[bot] commented 2 years ago

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

gencer commented 2 years ago

not stale

stale[bot] commented 2 years ago

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

gencer commented 2 years ago

not stale

stale[bot] commented 2 years ago

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.