Feature: plugins from local paths

[joscha]

We have a mono repository and I would like to write a few Buildkite plugins for internal things that are in this mono repository (a local stack that we start up in CI to do some e2e testing for example), hence I would like to co-locate the plugin code and the local stack code, but it seems as if the plugin only takes org/repo, not anything like /path/from/checkout/to/plugin.

[lox]

That's strange, it should work! Mind sending us a pipeline.yml that isn't working to hello@buildkite.com

[joscha]

@lox maybe my definition is wrong?

Does this look sane:

    plugins:
      /local_stack: ~

?

[lox]

At the minute our plugin gear makes the assumption that there is a git repository that can be cloned:

https://github.com/buildkite/agent/blob/master/bootstrap/bootstrap.go#L464

So this works for me:

steps:
  - label: "Test local plugin"
    agents:
      queue: "${BUILDKITE_AGENT_META_DATA_QUEUE:-default}"
    plugins:
      "/Users/lachlan/Projects/buildkite/docker-compose-buildkite-plugin": ~

Plugins that don't involve a git checkout are something we could consider, but not a thing yet.

[lox]

I think this might be tricky to make work, I can't think of a good way to indicate that it's a local plugin and doesn't need checkout. Possibly we could just infer it from relative paths? ./plugins/local_stack or similar.

[joscha]

At the minute our plugin gear makes the assumption that there is a git repository that can be cloned:

The plugin I am talking about is part of the current checkout, so inside a git repo, technically, but not at the root level

So this works for me:

That looks like what I want, but without the global path. I suppose I could try:

plugins:
  "${BUILDKITE_BUILD_CHECKOUT_PATH}/local_stack"

which I would expect to load a plugin from the local checkout.

[lox]

Yeah, special casing this for sub-dirs of BUILDKITE_BUILD_CHECKOUT_PATH would probably make sense. Thoughts @toolmantim?

[joscha]

Possibly we could just infer it from relative paths?

I expected a prefix of / to denote the current checkout path, e.g.

plugins:
  "/bla"

would load a plugin from ${BUILDKITE_BUILD_CHECKOUT_PATH}/bla

[lox]

To me that would be an absolute path from the system root. I'd expect ./blah to be what you'd want.

[joscha]

I'd expect ./blah to be what you'd want.

👍 fine with me, that would allow build agents to provide plugins globally as well if they need to.

[toolmantim]

Local plugins! Interesting idea.

For syntax, ./plugins/some-local-plugin sounds like the best bet to me, only because I would have expected /plugins/some-local-plugin to be an explicit filesystem path (not relative to the checkout)

Wondering through the other implications… I guess the version string part of the plugin key (./plugins/some-local-plugin#v1.0.0) would be invalid? And it wouldn't need to check out the plugin.

@keithpitt designed the initial naming and git repo path conventions. He might have some ideas too.

[toolmantim]

(Oh sorry, I confused this with local plugins in an actual build repo. Nevermind me)

[joscha]

This has been implemented via https://github.com/buildkite/agent/pull/878

[lox]

Let us know how you go with it @joscha! I'd forgotten about this issue, apologies, I should have referred back to it!

[joscha]

Let us know how you go with it @joscha! I'd forgotten about this issue, apologies, I should have referred back to it!

no worries, I just saw it in the release notes and remembered.