Everything is a package rule

[rarkins]

There are existing issues this is partly a duplicate of, but I wanted to start a clean feature request discussion.

I think that internally, Renovate should adopt the concept that "everything is a package rule". All other nested approaches of configuration (e.g. "major": { "enabled": true } or "python": { "masterIssue": true } should be considered as kind of "shortcuts" or syntactic sugar that are immediately massaged into packageRules.

Of course, packageRules are evaluated in their array order, so the order of such shortcuts massaging is important. I think that the order we massage them into the packageRules array should be logical, but if ever in doubt then users should simply use the longer-form packageRules in the exact order they want, instead of shortcuts.

What type of "shortcuts" exist today?

1. Languages e.g. python or docker
2. Managers, e.g. pip_requirements
3. DepTypes, e.g. devDependencies and engines
4. Update types, e.g. major

Missing today: Datasources

The next question is how to order them. Let's say there are in fact 7 different sources for the final packageRules, in the order I think they need to be applied:

1. The config's explicitly defined packageRules
2. All non-nested (top-level) config, e.g. "semanticCommits": false
3. languages
4. managers
5. depTypes (maybe)
6. datasources
7. updateTypes

I haven't considered how to handle config and rules within presets either, but I think they need the same "flattening" logic applied and then be put first so that everything in config can potentially override anything in presets.

e.g. take this config:

{
  "semanticCommits": false,
  "docker": {
    "pinDigests": false
  },
  "npm": {
    "major": {
      "enabled": false
    }
  },
  "rubygems": {
    "semanticCommits": true
  },
  "packageRules": [{
    "packageNames": ["lodash"],
    "enabled": false
  }]
}

You'd get:

{
  "packageRules": [{
    "packageNames": ["lodash"],
    "enabled": false
  },
  {
    "packagePatterns": ["*"],
    "semanticCommits": false
  },
  {
    "languages": ["docker"],
    "pinDigests": false
  },
  {
    "managers": ["npm"],
    "updateTypes": ["major"],
    "enabled": false
  },
  {
    "datasources": ["docker"],
    "pinDigests": false
  },
  {
    "datasources": ["rubygems"],
    "semanticCommits": true
  }]
}

Recall that the "final" packageRules like the above is intended to be understandable although not necessarily simple/optimized.

Challenges:

• I think we may want to deprecate depTypes shortcuts. When they were added, we supported only npm, and we have not kept up with the various depTypes that we support with other managers. So either we maintain a single array of allowed depTypes (across languages) and continue supporting them with shortcuts, or we deprecate them and add config massage rules to move them to packageRules as part of config massage, later to be deprecated.
• Datasources often have the same name as languages or managers, e.g. docker. I think today when people configure "docker" as a rule, a datasource rule would be more applicable than a language rule. This has caused problems because managers like Ansible do not semantically inherit from the "docker" language because they are multi-language. For cases where languages and datasources are the same, should we create two packageRules each? e.g. one for language and another for datasource?

---

edit: fix json typo

[rarkins]

Another impact of this: we do not need to define every language/manager/datasource within lib/config/definitions. Maybe we only define them if they need custom config.

I even have some doubts if we need to put each manager's custom config - most of which is highly recommended to never be overridden - within the definitions, or just define it within source code in lib/managers.

[HonkingGoose]

From what I gather, this would be a big potentially breaking change? At least it seems to me that this is a totally different concept than what Renovate uses now?

Do we still want to make such a big sweeping change?

I'm asking because this issue is from Oct 2019, and the code base has probably changed a lot in that time. 😄

I noticed we have some related issues linking back to this one, so might want to look at that first to decide whether or not this is still relevant?

[rarkins]

From what I gather, this would be a big potentially breaking change?

Big change to our config logic: yes

Potentially breaking: it would change the way we resolve configs, fixing many things which produce confusing behaviour today but also changing some behaviour that people may have been relying on.

At least it seems to me that this is a totally different concept than what Renovate uses now?

Not at all. The end result will be that most cases function exactly as they were

Do we still want to make such a big sweeping change?

Yes, and we must

I'm asking because this issue is from Oct 2019, and the code base has probably changed a lot in that time. 😄

I noticed we have some related issues linking back to this one, so might want to look at that first to decide whether or not this is still relevant?

It's a lot of work, and we haven't decided how to handle the backwards-incompatible aspect of it for users of the app, who can't control when they take a major release.

[rarkins]

A way forward: I think we might be able to implement this as a feature flag - potentially even hidden at first - and opt ourselves into it as test subjects first.

[rarkins]

Here's how to convert a config into all packageRules.

First of all, there's going to be some certain config which can't go into packageRules, such as hostRules, which I'm going to ignore for now.

function configToRules(input):

output = { packageRules: [] }
// Add extends rules first
if input.extends:
  output.packageRules = configToRules(resolve(input.packageRules))
  delete input.extends
// now add top-level config
let rootRule = { matchAll: true }
output.packageRules.push(rootRule)
for (key, value) of input:
  if key === 'packageRules':
    // concatenate them to output.packageRules
    // now reset the rootRule to come after those
    rootRule = { matchAll: true }
    output.packageRules.push(rootRule)
  else if value is an object:
    nestedRule = {}
    // add appropriate matching based on the key name (e.g. minor, or npm, etc)
    nestedRule = { ...nestedRule, ...value }
    output.packageRules.push(nestedRule)
  else:
    rootRule[key] = value

[rarkins]

Summarizing:

• new hidden config option, resolveToRules, defaults to false
• All extends will be converted first (therefore take lower priority)
• Top-level config (e.g. "labels": ["deps"]) get converted next
• Nested config objects (such as "npm": { "labels": ["foo"] } are converted last
• Any existing packageRules get added in the same order they appeared in the input config