search

buildkite / feedback

Got feedback? Please let us know!
https://buildkite.com
25 stars 24 forks source link

Support different git repository per pipeline step. #318

Open buchgr opened 6 years ago

buchgr commented 6 years ago

Hi!

We are the team building Bazel (https://bazel.build). We want to have a pipeline that builds a Bazel binary at a specific commit, then takes this Bazel binary and checks out a different repository (i.e. Tensorflow) and tries to build and test it using this Bazel binary. Something like this:

steps:
  - label: "Build Bazel"
    command:
      - bazel build //src:bazel
      - buildkite-agent artifact upload bazel-bin/src/bazel

  - wait:

  - label: "Build TensorFlow"
    git_repository: https://github.com/tensorflow/tensorflow.git 
    command:
      - buildkite-agent artifact download bazel-bin/src/bazel /tmp/dir/ -step "Build Bazel"
      - /tmp/dir/bazel-bin/src/bazel build ...

We are currently working around this limitation by cloning TensorFlow in our own scripts and managing these repositories ourselves. However, it would be great if Buildkite could do that for us!

Please let me know what you think. Thanks!

gtirloni commented 6 years ago

Having just read something about this, maybe creating two pipelines and trigger the TensorFlow one (while passing some env var about where to locate the Bazel binary)? I haven't tested this so I'm not sure how (or if) they would share the build workspace/directory.

phraemer commented 4 years ago

Does anyone have any tips on how to achieve this?