search

renovatebot / renovate

Home of the Renovate CLI: Cross-platform Dependency Automation by Mend.io
https://mend.io/renovate
GNU Affero General Public License v3.0
17.19k stars 2.24k forks source link

Default org config #22212

Closed rarkins closed 6 months ago

rarkins commented 1 year ago

What would you like Renovate to be able to do?

I would like to support the idea of default settings for an org, without it needing to be explicitly extended using presets.

Although using a central preset is the most transparent way to have shared org settings, it also then mandates the use of a repository config file. Instead, we should support the ability to set org default settings and use them if present.

If you have any ideas on how this should be implemented, please tell us here.

Existing/Related

Today we have a related capability when it comes to onboarding. For an org like my rarkins on github.com, we'd look for:

I know that it's a little confusing with default.json but I don't have a time machine to go back and change that decision, so all we can do is make the best decision now for the new capability.

Performant

We need to avoid adding too many API calls because of this.

If we check for this preset on every run, we shouldn't support more than one possible repo/file name combination, so that we cap at one API call per repo run. In such a case I suggest it should be a file named like org-default-config.json in e.g. github.com/rarkins/renovate-config.

Alternatively we could aggressively cache it, e.g. once per run, which would mean any cascading checks at least happen only once. Such caching would also need to be possible in the hosted Renovate app somehow. One possibility for that would be that the hosted app doesn't support the org default directly in Renovate (disables this new feature) but instead watches for changes to this magic org/file and populates the config automatically with each job.

UX

The downside of this capability is that repos will have some config applied to them which is not explicitly "documented" in the repo - even if a repo config is present. We could make it more transparent by:

Is this a feature you are interested in implementing yourself?

Maybe

viceice commented 1 year ago

don't we already dropped renovate.json fallback? if not we should do it in next major. it's at least deprecated with a warning since some majors.

secustor commented 1 year ago

I like the idea.

Regarding caching we should be fine caching the organization settings for an hour.

lmilbaum commented 1 year ago

What if the codebase tracked in multiple orgs or for that matter in multiple git forges/servers?

rarkins commented 1 year ago

In each of the SCM platforms we support, there's only ever one top-level org. GitLab can have groups and subgroups, and ideally we'd support that in a "cascading" matter.

I don't understand how a repo can have multiple "git forges/servers"

viceice commented 1 year ago

i don't need this, i prefer central presets to have more visibility.

Shegox commented 1 year ago

One thing, which might be worth supporting (whatever filename/location) we choose is an explicit opt-in to use this. E.g. a preset, which should be automatically extended in side the org must have the configuration setting autoExtendThisConfigForOrganization: true (name tbd).

This would have two advantages:

rarkins commented 1 year ago

Let's assume we have a repo/file name combination which is next to impossible to conflict with accidentally when creating presets. In that case it's naturally opt-in. @Shegox can you clarify your proposal/requirement in that context?

Shegox commented 1 year ago

If it is a novel/new repo/filename name combination then you can ignore my comment. It was more if we would reuse $ORG/renovate-config/default.json or so as name.

lmilbaum commented 1 year ago

In each of the SCM platforms we support, there's only ever one top-level org. GitLab can have groups and subgroups, and ideally we'd support that in a "cascading" matter.

I don't understand how a repo can have multiple "git forges/servers"

What I meant was that a product can be consisted of repos that are tracked in multiple GH orgs/GL namespaces/git servers. For example:

rarkins commented 1 year ago

Right now we're just focused on the repo -> org config mapping, so I'll hide these comments as off-topic

jamietanna commented 1 year ago

I would +1 being able to have org-wide defaults that can be inherited by others. It's something I've implemented in our org, and my blog post around it seems to be getting several hits a week from what I can see, which may mean there's some others wanting it too.

To confirm, is this more for hosted Renovate, or also for self-hosted? In the self-hosted world it can be easier to set global configuration that everyone inherits, but it would be more convenient to do this on the org level than the global self-hosted runner

rarkins commented 1 year ago

In self-hosted the need for this is less so, because people typically have one org and they can put default settings in global config. However I would intend this feature to work in both self-hosted and hosted.

rarkins commented 11 months ago

We could make it org-default-config.json5 to allow people to use json5 features if they wish

rarkins commented 6 months ago

We could use the term "inherited" config seeing as we don't use that term yet. e.g. org-inherited-config.json5.

rarkins commented 6 months ago

The next challenge we have concerns performance and API use. Let's say we turn this feature on in the hosted app which in total has around a million repos installed. On day one, no orgs will have this feature/file. This means we'll end up doing millions of 404 API calls for the file, all of which count towards API quota - i.e. 1 extra API call consumed per repo job.

One possibility:

If we wanted to be really advanced we could potentially add logic where we only enable this feature if there's a renovate-config repo existing and installed into the org, as this would reduce the traffic down a lot. We could maybe even try looking for the particular filename itself but that could be complex.

Another benefit of this org inherited config is that it could allow file-based control over additional features in hosted environments such as onboarding and requireConfig. Then such controls aren't really "globalOnly" controls anymore and instead org-level config.

rarkins commented 6 months ago

We also need to think about the implications of public vs private. If an org config is private then public repos won't be able to access it. So it would need to be public unless every repo in the org is private.

renovate-release commented 6 months ago

:tada: This issue has been resolved in version 37.267.0 :tada:

The release is available on:

Your semantic-release bot :package::rocket:

jamietanna commented 6 months ago

Awesome 👏🏽 looking forward to giving this a go with the hosted app, when it lands 😁

rarkins commented 6 months ago

We need to add a feature to that backend allowing either (a) manual enabling, or (b) automatic enabling based on webhooks (i.e. adding/editing the magic filename)

pietrodn commented 4 months ago

Hi, is this available also on the GitHub hosted app?

rarkins commented 4 months ago

Not yet, please stay tuned.