Feature Request: support ops-files from multiple repositories in bosh-deploy

[aashah]

Not sure whether concourse will fully be able to support this, but it'd be nice to allow teams to have their own ops files. Currently, in order to write our own, we had to "vendor" the ops-files in cf-deployment.

[cf-gitbot]

We have created an issue in Pivotal Tracker to manage this:

https://www.pivotaltracker.com/story/show/139854967

The labels on this github issue will be updated when the story is started.

[dsabeti]

@aashah Yeah, this one is hard, because concourse doesn't support making tasks with a variable number of arguments.

One thing we talked about shipping was a task that copies ops files into a centralized location. So imagine that you had the cf-deployment repo and a xena-env repo that had custom ops files in a directory called custom-ops-files. You could configure your job like this:

- name: deploy
  plan:
  - get: cf-deployment
  - get: xena-env
  - task: centralize-ops-files
    file: cf-deployment-concourse-tasks/centralize-ops-files/task.yml
    input_mapping:
      input-ops-repo: cf-deployment
      target-ops-repo: xena-env
    output_mapping:
      updated-target-ops-repo: updated-xena-env
    params:
      INPUT_OPS_FILE_PATH: operations/gcp.yml
      TARGET_OPS_FILE_PATH: custom-ops-files
  - task: bosh-deploy
    file: cf-deployment-concourse-tasks/bosh-deploy/task.yml
    input_mapping:
      ops-files: updated-xena-env
    params:
      OPS_FILES: "custom-ops-files/gcp.yml custom-ops-files/xena-specific-ops-file.yml"

The task would be REALLY simple (probably just a cp from one directory to another), so you could implement this yourself.Also, you'd have to run this task once for each ops file that you wanted to centralize. This isn't a great solution. Unfortunately, I'm not sure I can think of anything better.

Let me know if this would satisfy your use case. I'll also wait to see if other teams need something like this as well. In the meantime, I can try to poke the concourse team for help.

[medvedzver]

I think ticket can be closed. You can specify

params:
      OPS_FILES: "file1" "file2" "file3"

according to this

[aashah]

Hey @medvedzver,

That only works for files that are in the same ops-files input. In our case, we have additional ops files located in a different git repository. We'd also like to use the ops-files located in cf-deployment. This is not possible, as only one resource can be bound to the ops-files input.

[medvedzver]

My bad. You are right.

[dsabeti]

Hey @aashah, we have a new story for a task that collects the ops-files from two repos into into a single input for bosh-deploy tasks. If you have more than two repos, you can always re-run this task to append more ops-files to the list.

[dsabeti]

Hey @aashah and @medvedzver, take a look at this new task. This should allow you to "collect" all of your ops-files into a single directory so that your bosh-deploy task can use it.

I'm going to close the issue, but feel free to reopen it if you have other questions.