Define custom dependencies from yaml file

[machine424]

Thank you for this great tool.

What would you like Renovate to be able to do? Suppose I have a yaml file (or any file) with a key:

...
dep-version: v1.0.0
foo: bar
...

Is there a way to tell (teach) Renovate to watch dep (let's say a python package) releases and update the key's value ? Can we manually define dependencies for Renovate ? Can renovate parse the file and detect that dep-version's value correspond to sth that can be updated ?

If it's not already the case and you would like supporting this, I could help implementing.

[rarkins]

It sounds like the regex manager we added should support this. Please check it out both in the docs as well as by searching past issues in this repo. https://docs.renovatebot.com/modules/manager/regex/

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed soon if no further activity occurs.

[machine424]

Thanks for your answer. Yes, the regex manager does exactly what I need, but identifying a value using a regex in a yaml file may be hard and error prone, couldn't we support more languages/syntaxes depending on the format of the file, for example using JSON Pointer (RFC 6901) for JSON/YAML files. (how about Custom Manager Support using JSON Pointer)

[viceice]

json-pointer can maybe also help for json based files

[rarkins]

The following would help:

• An example file (JSON, YAML, etc)
• How you would propose the configuration syntax would work (e.g. using regex manager as inspiration)

[machine424]

For a ci-file.yaml file:

job-name:
  variables:
    DEP_VERION: v1.0.9

to keep DEP_VERSION up-to-date for example:

{
  "jsonPointerManagers": [
    {
      "fileMatch": ["^ci-file.yaml$"],
      "matchValues": ["/job-name/variables/DEP_VERION", "maybe other paths"],
       # configure the datasource (predefined or custom)
    }
  ]
}

I don't know how Renovate patches json/yaml files, but we can use JSON-Patch (RFC 6902) plugins that are based on JSON Pointer to do so.

For datasources, I don't know if we can define custom ones (through configuration templates) as for regexManager to avoid adding comments to the yaml file. There is an opened issue about tha: https://github.com/renovatebot/renovate/issues/4674

[rarkins]

Patching can be done automatically once extraction is done.

Datasource changes should be out of scope for this discussion.

Ideally this would have the capability to match multiple values like the regex manager can. The example right now is a little too simple to plan with. Are there no real file types in mind?

[machine424]

I know patching will be done by Renovate, I just think that it will be easier if a JSON-Patch plugin was used by Renovate, but this is not a problem.

A real use case may be a .gitlab-ci.yaml file whose jobs' dependencies should be kept up to date.

variables:
  DEP3_TAG: v2.0.8
job-name:
  image: img:$DEP3_TAG
  variables:
    DEP1_VERSION: v1.0.9
    DEP2_VERSION: v2.0.0
  script:
    - wget --no-verbose https://github.com/../download/$DEP1_VERSION
    - command --use-version $DEP2_VERSION

we don't want to maintain a custom docker image that adds:

    - wget --no-verbose https://github.com/../download/$DEP1_VERSION
    - command --use-version $DEP2_VERSION

into img:$DEP3_TAG (even if the regex manager can help us), we want to keep the dependencies management in one place.

this choice, gives also to the user the freedom and flexibility to easily override a dependency without having to build a new docker image.

[rarkins]

I've looked at JSON pointer and JSONpath and don't fully like either. Unfortunately I don't see any other feasible approaches yet.

Here's my thinking so far:

• The goal of this is to have a custom manager similar to regex manager but which lets you operate on a JSON structure as input instead of raw text
• We could support not just JSON but also well-known formats like YAML and TOML. Parsing could be done automatically based on file extension but we could also support a config override to specify the parser type
• The function should return an array of extracted dependencies and ideally not need you to configure one manager per dep or dep type

A good aim would be that this manager would be capable of extracting a basic package.json (e.g. including dependencies and devDependencies) although it could be a little challenging.

I'm also thinking we lean heavily on handlebars templating again.

[brad-jones]

What would be really cool is if renovate supported some kind of plugin based architecture. Maybe based on gRPC similar to how hashicorp tools work or even just a javascript module loaded at runtime. So that one could add custom / non-standard package managers to renovate without having to fork the project.

[rarkins]

I'm going to close in favor of #15193