Pipeline resource fly version must match Concourse deployment

[hstenzel]

Since the concourse-pipeline-resource image contains the fly binary and does not fly sync, this leads to a failure of the resource when either the resource's fly version is ahead of the Concourse deployment or whenever the Concourse deployment is ahead of the concourse-pipeline-resource.

The workaround for this is to use a specific version of the resource by specifying the docker tag, but then the pipeline configuration is required to be changed just because the Concourse version is upgraded.

It would be better if the resource did a fly sync internally to be sure that it is using the correct version of fly for the Concourse deployment. Even better if the resource could just have a number of versions of fly internally & call an api to determine which it should use.

[eedwards-sk]

it almost looks like there is some behavior to perform a fly sync in the code for Login:

https://github.com/concourse/concourse-pipeline-resource/blob/45fa8ca3d8a31898e83eb4420a2c700b92eabe21/fly/fly.go#L72

maybe there's a bug in that behavior?

[ukabu]

It looks like the sync is done after login instead of before. No option is also specified for the sync (-c concourse-url).

[quickjp2]

Currently, fly sync is being ran, but this doesn't allow the resource to be used by different major versions (e.g., target is 3.14.1 and the current baked version is 5.3).

What will need to be implemented is a pre-check if there is a discrepancy between the target's major version and the current fly cli's major version, and "sync" if there is via the api.

If needed, I have implemented the feature in my fork. I'm not very good at unit tests with golang, and I'm currently not submitting a PR until I can get some testing in first. However, you can still build the container, push it into a private registry, and have concourse reference that.

[vito]

FWIW I'm intending to fix this problem by just introducing a native set_pipeline step to Concourse: https://github.com/concourse/rfcs/pull/31

Ultimately I would much rather deprecate this resource type. It's too awkward to support. Once we've far enough along on our Core roadmap I suspect most people won't need it anymore.

[irbekrm]

@hstenzel how did you manage to specify the resource type? We are running into the same issue when setting some pipelines on a Concourse v4.X.X. I have tried to specify docker tag to point to a version of Concourse pipeline resource with fly v4.X.X like so

- name: concourse_pipeline
  type: docker-image
  source:
    repository: concourse/concourse-pipeline-resource
    tag: 1.2.0

but the resource still comes up with the latest version of fly.

[hstenzel]

Sorry, I'm in a different position and can't check that detail.

[ZiadSalah]

@irbekrm Hi. I realise you posted your question a while ago. I struggled with the same problem today but eventually got it to work by calling fly -t <insert target> check-resource-type --resource-type <insert resource type>

Thought I'd post this if you still needed help or someone else with a similar issue sees this.

[irbekrm]

Thanks @ZiadSalah ! We don't have a need to pin the resource anymore, but this might become useful at another time.