Support git-refs for jsonnet-bundler

[patoarvizu]

What would you like Renovate to be able to do?

Some popular jsonnet dependencies like Grafana's jsonnet-libs aren't tagged/versioned so downstream applications usually just point to master. Renovate fails to update these dependencies, with the error message Dependency jsonnet-libs has unsupported value master, and "skipReason": "invalid-value". Another use case for this is an internal library that has a very frequent release cycle but it's used in many applications, so rather than updating multiple repos multiple times a week, service owners are encouraged to just point to the main branch.

I'm fairly new to renovate, but my understanding is that the problem is that the jsonnet-bundler manager only supports git-tags as the data source, and master, main, etc., are not git tags but a refs.

(I understand that having an unpinned dependency like main is a bad practice and an anti-pattern but in some cases, like jsonnet-libs, there is no practical alternative.)

If you have any ideas on how this should be implemented, please tell us here.

I have attempted to use postUpgradeTasks to run jb update to force an update of the jsonnetfile.lock.json but it doesn't seem to be running the command. I'm assuming this is because it fails to detect a new dependency (given that master isn't a valid tag), so it reports no updates and renovate doesn't run any of the postUpgradeTasks commands.

Is this a feature you are interested in implementing yourself?

No

[github-actions[bot]]

Hi there,

Help us by making a minimal reproduction repository.

Before we can start work on your issue we first need to know exactly what's causing the current behavior. A minimal reproduction helps us with this.

To get started, please read our guide on creating a minimal reproduction to understand what is needed.

We may close the issue if you (or someone else) have not provided a minimal reproduction within two weeks. If you need more time, or are stuck, please ask for help or more time in a comment.

Good luck,

The Renovate team

[patoarvizu]

I created this repo with a jsonnetfile.json file, a Renovate config, as well as instructions on how to run a minimal self-hosted renovate on a local k3d cluster.

Would this be enough?

[rarkins]

Please let me know if I've understood correctly?

• You don't want that value of master to be changed
• You would like Renovate to update the sha value in the lock file

If so, the next step is that we need to know what steps/commands can be used to bump just one dependency at a time. Presumably jb update would bump everything.

[rarkins]

Actually I just realized that your reproduction doesn't contain a lock file, so something doesn't seem right either in my understanding or in your reproduction

[patoarvizu]

Ah sorry, I missed that. I just updated the repo with a lock file.

The jsonnetfile.json is pointing to the master branch of https://github.com/grafana/jsonnet-libs, but the version in the lock file is the sha1 of an upstream commit on January 24. This is intentional, to simulate a dependency that should be pointing to the latest but the lock file is behind.

Effectively, what I'm looking for is a way for Renovate to update the lock file to point to the most recent commit, without changing the value of master on jsonnetfile.json.

My hypothesis here is that renovate sees master as an invalid version ("skipReason": "invalid-value") and just skips it entirely, including the lock file maintenance.

jb can update one dependency at a time with jb update <git-remote>/<subdir>, e.g. if the jsonnetfile.json looks like this:

{
  "version": 1,
  "dependencies": [
    ...
    {
      "source": {
        "git": {
          "remote": "https://github.com/grafana/jsonnet-libs",
          "subdir": "ksonnet-util"
        }
      },
      "version": "master"
    }
    ...
  ]
}

Then jb update https://github.com/grafana/jsonnet-libs/ksonnet-util will update the lock file for just that dependency.

I hope that clarifies!

[rarkins]

Specifically this means supporting rangeStrategy=update-lockfile for jsonnet-bundler, which would cover all dependency types, not just git

[patoarvizu]

I believe so, at least based on what I'm reading in the documentation (I'm still fairly new to Renovate).

[patoarvizu]

Also, I think I found a separate issue while I was trying to make this work. I can open a separate ticket but wanted to include it here because it may be relevant to the implementation too.

I went outside of my initial use case and started using git labels for dependencies on private repos. Renovate can correctly detect the updated dependencies and modify the jsonnetfile.json file, but when it tries to update the lock file it gets a git error along the lines of:

fatal: could not read Username for 'https://my.private.ghe.com': No such device or address
fatal: could not read Username for 'https://my.private.ghe.com': No such device or address
jb: error: updating: downloading: exit status 128

Even though I do have a host rule for my.private.ghe.com (otherwise Renovate wouldn't be able to clone my private repo, nor to detect the upstream dependencies). I suspect that this is happening because to update the lock file, the jsonnet-bundler manager shells out to run jb update https://my.private.ghe.com/my-private-repo and it has no control over the credentials used by jb.

@rarkins Would you prefer if I open a separate ticket for this, or do you want to track it here?

[rarkins]

Separate feature request: private registry support for jb lock files

[patoarvizu]

Done! 👍

[rarkins]

I think there are three separate use cases here:

1. A git ref where there's a version in the URL itself (e.g. https://github.com/org/repo#v1.1.0). In this case we want to update the package file and then refresh lock file accordingly. It would be a patch, minor or major version update
2. A git ref where there's no version in the URL (e.g. https://github.com/org/repo or https://github.com/org/repo#next). In this case it's a lockfile-only digest update
3. A git ref where the has is in the package file (e.g. https://github.com/org/repo#a8asf21cc). In this case it's a package file digest update

It may be easiest to implement these one by one, with the version update being easiest.