InvalidAuthentication error intermittently when using correct creds

[ctaymor]

We are using the semver resource with a GCS bucket. We are seeing intermittent, frequent failures to PUT the semver resource. However, it succeeds sometimes. We started seeing this two days ago, and 10 out of the last 18 builds have failed for this reason. (and out of the last 11 builds, 8 failed) This is adding significant flakiness to our pipeline.

We get the following error during the put (not during the subsequent get):

error bumping version: InvalidAuthentication: The provided authentication header is invalid.
    status code: 400, request id: , host id: 

Our resource is configured as follows:

- name: release-version
  type: semver
  source:
    key: release-version
    access_key_id: ((gcs-access-key-id))
    secret_access_key: ((gcs-secret-access-key))
    bucket: our-pipeline-store
    region_name: us-east1
    endpoint: storage.googleapis.com

We're doing a put with build params:

- put: release-version
      params: {pre: build}

We are using concourse version 3.8.0.

[vito]

I don't think we'll be able to do much with this. :/ Are there more logs available from GCS? I mean, we're just passing along the token, if they think it's invalid (sometimes?) there's not much we can do.

Have you considered using the gcs driver? It has a different auth setup:

https://github.com/concourse/semver-resource/blob/master/README.md#gcs-driver

[ctaymor]

I know there's insufficient info here. :(

I was hoping if other folks were seeing it, there might be solution others had seen, or further information about getting more logging info out of the resource.

I'll look into the gcs driver and see if our gcs logs our helpful. Thanks.