search

adikari / monorepo-diff-buildkite-plugin

Run separate pipelines for each folder in your monorepo
MIT License
202 stars 78 forks source link

Add an option to use a pre-downloaded binary vs downloading it every time #68

Closed rajatvig closed 2 years ago

rajatvig commented 3 years ago

The plugin downloads the binary every time the build is run which takes time. Instead could there be an option to assume the binary is in /usr/bin?

adikari commented 3 years ago

@rajatvig the executable is 3 MB so the download should not add any significant time to your builds. At least that's the case for me but I am not sure about your scenario. It literally downloads the binary in 1 second.

image

adikari commented 3 years ago

I am also not sure how you are running your builds. Are you planning to download the binary manually at chuck it in a directory on your build machine? To me, this feels like optimization with not much to gain. Let me know if you think otherwise or details of your scenario.

rajatvig commented 3 years ago

@adikari Thanks for looking at the issue. Will update the PR as commented soon.

There are two issues

  1. We run too many builds a day and every build downloading the binary is wasteful considering the binary does not actually change.
  2. Downloading binaries at build time is a security risk for us. We have disabled all plugins other than approved ones.

With regards to the question on packaging the binary, we are baking in a custom docker image and a compute image with the agent and this binary and running it as a container or VM varying on what the build needs to do.

Hope that explains the issue better