Using git commit refs in kustomize

[daurnimator]

For kustomize remote references, we pin to upstream commit hashes, e.g. https://github.com/hashbang/gitops/blob/d22d1c48826d625321d9b86be7d67bdb09fb2f5a/mtls/kustomization.yaml#L5

renovate seems to try and check this as a tag: https://app.renovatebot.com/dashboard#github/hashbang/gitops/271823778

DEBUG: Dependency drGrove/mtls-server has unsupported value 705e1c2a5f193f1d309af4c6e0124b9096c2dce9
DEBUG: GitHub 404
{
  "url": "https://api.github.com/repos/drGrove/mtls-server/git/refs/tags/705e1c2a5f193f1d309af4c6e0124b9096c2dce9"
}
DEBUG: Error getting tag commit from GitHub repo
{
  "githubRepo": "drGrove/mtls-server",
  "err": {
    "name": "HTTPError",
    "timings": {
      "start": 1607480889510,
      "socket": 1607480889510,
      "lookup": 1607480889510,
      "connect": 1607480889517,
      "secureConnect": 1607480889528,
      "upload": 1607480889528,
      "response": 1607480889710,
      "end": 1607480889710,
      "phases": {
        "wait": 0,
        "dns": 0,
        "tcp": 7,
        "tls": 11,
        "request": 0,
        "firstByte": 182,
        "download": 0,
        "total": 200
      }
    },
    "message": "Response code 404 (Not Found)",
    "stack": "HTTPError: Response code 404 (Not Found)\n    at Request.<anonymous> (/home/ubuntu/renovateapp/node_modules/renovate/node_modules/got/dist/source/as-promise/index.js:117:42)\n    at runMicrotasks (<anonymous>)\n    at processTicksAndRejections (internal/process/task_queues.js:97:5)",
    "options": {
      "headers": {
        "user-agent": "Renovate Bot (GitHub App 2740)",
        "accept-encoding": "gzip, deflate",
        "accept": "application/vnd.github.machine-man-preview+json",
        "authorization": "***********"
      },
      "url": "https://api.github.com/repos/drGrove/mtls-server/git/refs/tags/705e1c2a5f193f1d309af4c6e0124b9096c2dce9",
      "username": "",
      "password": "",
      "method": "GET",
      "http2": false
    },
    "response": {
      "statusCode": 404,
      "statusMessage": "Not Found",
      "body": {
        "message": "Not Found",
        "documentation_url": "https://docs.github.com/enterprise/2.18/user/rest/reference/git#get-a-reference"
      },
      "headers": {
        "date": "Wed, 09 Dec 2020 02:28:09 GMT",
        "content-type": "application/json; charset=utf-8",
        "transfer-encoding": "chunked",
        "connection": "close",
        "server": "GitHub.com",
        "status": "404 Not Found",
        "x-poll-interval": "300",
        "x-github-media-type": "github.v3; param=machine-man-preview; format=json",
        "x-ratelimit-limit": "5250",
        "x-ratelimit-remaining": "5241",
        "x-ratelimit-reset": "1607483616",
        "x-ratelimit-used": "9",
        "access-control-expose-headers": "ETag, Link, Location, Retry-After, X-GitHub-OTP, X-RateLimit-Limit, X-RateLimit-Remaining, X-RateLimit-Used, X-RateLimit-Reset, X-OAuth-Scopes, X-Accepted-OAuth-Scopes, X-Poll-Interval, X-GitHub-Media-Type, Deprecation, Sunset",
        "access-control-allow-origin": "*",
        "strict-transport-security": "max-age=31536000; includeSubdomains; preload",
        "x-frame-options": "deny",
        "x-content-type-options": "nosniff",
        "x-xss-protection": "1; mode=block",
        "referrer-policy": "origin-when-cross-origin, strict-origin-when-cross-origin",
        "content-security-policy": "default-src 'none'",
        "vary": "Accept-Encoding, Accept, X-Requested-With",
        "content-encoding": "gzip",
        "x-github-request-id": "B22A:2CFE:12CB0D1:1766A5E:5FD03639"
      },
      "httpVersion": "1.1"
    }
  }
}
DEBUG: Could not getDigest
{
  "newValue": "705e1c2a5f193f1d309af4c6e0124b9096c2dce9"
}

How can we get renovate to check that we are using the ref of the latest tag?

[viceice]

This is currently not supported. This would require a new datasource github-refs and some fixed handling in kustomize manager.

You can try to use a regex manager as workaround and disable kustomize for this dependency.

[viceice]

Blocked by #7877

[daurnimator]

This is currently not supported. This would require a new datasource github-refs and some fixed handling in kustomize manager.

Why would this be github specific (aside from a possible optimization): you could:

• clone the repo
• describe old commit hash: notice that there is a tag that points at it
• observe that there is a newer tag
• describe the newer tag to get commit hash
• replace old commit hash with new commit hash in the file.

[viceice]

This can maybe already done with regex manager, try something like:

kustomization.yaml

resources:
  - github.com/drGrove/mtls-server/manifests/base?ref=705e1c2a5f193f1d309af4c6e0124b9096c2dce9 # tag=v0.17.0

renovate.json

"regexManagers": [
  {
    "fileMatch": ["(^|/)kustomization.yaml$"],
    "matchStrings": [
        "- github\\.com\\/(?<depName>[^/]+?\\/[^/]*?)\\/.*\\?ref=(?<currentDigest>[a-f0-9]{40}) # tag=(?<currentValue>.*)"
      ],
    "datasourceTemplate": "github-tags"
  }
]

https://regex101.com/r/iIm9nm/1

This will also suggest digest updates if the tag is moved.

[lindhe]

Thank you, viceice, for posting that example!

I think some syntax may have changed since. I'm posting the config I use (based on your example) to upgrade my Argo CD kustomization, in case that helps someone. Works like a charm! 😊

{
  "$schema": "https://docs.renovatebot.com/renovate-schema.json",
  "customManagers": [
    {
      "customType": "regex",
      "fileMatch": ["^argocd/base/kustomization.yaml$"],
      "matchStrings": [
        ".*\\/argoproj\\/argo-cd\\/(?<currentValue>v\\d+\\.\\d+\\.\\d+?)\\/.*",
        ".*app\\.kubernetes\\.io\\/version: (?<currentValue>v\\d+\\.\\d+\\.\\d+?)\\n"
      ],
      "datasourceTemplate": "github-tags",
      "depNameTemplate": "argoproj/argo-cd"
    }
  ],
  "packageRules": [
    {
      "matchPackageNames": ["argoproj/argo-cd"],
      "matchManagers": ["custom.regex"],
      "commitMessageTopic": "{{depName}}"
    }
  ]
}