search

dependabot / dependabot-core

🤖 Dependabot's core logic for creating update PRs.
https://docs.github.com/en/code-security/dependabot
MIT License
4.73k stars 1.02k forks source link

Allow configuration for how old dependencies must be to get updated #4715

Closed maciejmakowski closed 1 year ago

maciejmakowski commented 2 years ago

We'd like to have the ability to configure how old dependencies that are being bumped should be before they are considered "ready for update".

For example, just for illustration:

This would allow us to stagger updates and test out new dependencies across our libraries and services in a safe order.

As an example for what has happened just today for us: Production services picked up a new dependency update in the morning that was just released yesterday which turned out to be incompatible with an internal library. We did not have a chance to test that update with said library itself and catch the issue before production services started using it.

As an added bonus: We'd like to be able to configure that "allowed/minimum age" of a dependency in combination with the source it is coming from (we're mostly using Ruby):

  1. for external libraries coming from rubygems.org vs.
  2. for internal libraries coming from our internal gems repository
jeffwidman commented 2 years ago

Do you auto-merge PR's? Otherwise how is this getting released into production without a human authorizing it?

Personally, I prefer to get the PR opened so that myself/my coworkers are immediately notified of the update and then merge when ready. Maybe it fixes a critical CVE or bug? We probably wouldn't realize til we saw the Dependabot PR description with the release notes.

If you're trying to guard against a human being overly merge-happy, then could you hack together a github action that checks the author of the PR and if it's Dependabot, it blocks merge until X time has passed since the PR was opened? I haven't played with actions enough to know whether this would be trivial or impossible.

Technically the Dependabot PR may be opened later than the dependency release date, but in most situations it should be a good enough proxy that it'd work... and certainly much simpler to use the date the PR was opened than to try to introspect the release date from the original source repo.

maciejmakowski commented 2 years ago

The idea with the Github action sounds interesting, though we'd have to spend extra time on adding logic into it depending on the source of the gem which Dependabot should already know about.

Overall, I just also wanted to highlight that the teams owning internal libraries and teams owning services or other libraries that depend on the first set of internal libraries are likely to be non-overlapping groups of people. We'd like to be able to create a general set of rules to try to minimize the risk - it's partially due to human error and partially due to lack of test automation.

And yes, we do auto-merge some Dependabot PRs, but we just had a series of issues with an internal library where an update in redis from 4.6.0 to 4.6.1 broke fundamental logging functionality in production. Those updates were merged manually. I'll spare you the details for various reasons, but we think that staggered updates could help with catching "freak incidents" like this.

Lastly, note that Dependabot often says it's updating library "abc" and then it sneaks in a whole bunch of additional updates that would be easier to become aware of and review if they were their own independent PRs. And so when you skim a dozen of PRs quickly for "sanity", you are at risk of missing a bump like that, especially one that looks harmless according to semver, but actually does have impact on a library that you don't have detailed insight into (see the redis bump above).

jurre commented 2 years ago

sneaks in a whole bunch of additional updates

I'd assume these are transitive dependencies in this case? If they are not, I'd love to see examples of this to figure out what's happening! If they are transitive deps, there is not much we can do to prevent them from being updated.

limonte commented 2 years ago

I would like to bump this thread.

When a vulnerability is introduced to an npm package, it takes at least several days to discover the vulnerability and to report the vulnerable release to security databases (npm, Snyk, dependabot, etc.)

Because of that reason, it would make sense for developers to use 3rd party dependencies with the version that matches these conditions:

FYI renovate has this feature: https://docs.renovatebot.com/configuration-options/#stabilitydays

jeffwidman commented 1 year ago

Closing as a duplicate of https://github.com/dependabot/dependabot-core/issues/3651.

Please upvote that issue if you want this... we take many inputs into consideration for prioritizing what we work on, but one of them is what gets upvoted.