Dependabot sometimes generates PRs with unsolvable requirements.txt files

[FredericDT]

Reference: https://github.com/HenryzhaoH/bupt-ncov-report-tgbot/pull/15

In which case, dependabot opened a pr https://github.com/HenryzhaoH/bupt-ncov-report-tgbot/pull/13 for upgrading urllib3 from 1.25.8 to 1.26.5, which conflicts with requests requirement.

[deivid-rodriguez]

For whoever takes this issue. A possible approach here is:

• Fully unlock the dependency in a isolated environment. So in the example given

diff --git a/requirements.txt b/requirements.txt
index 9d6383d..b8e05ab 100644
--- a/requirements.txt
+++ b/requirements.txt
@@ -15,4 +15,4 @@ requests==2.23.0
 six==1.14.0
 tornado==6.0.3
 tzlocal==2.0.0
-urllib3==1.25.8
+urllib3

• Run pip-compile on that temp file:

pip-compile -U -P urllib requirements.txt -o upgraded_requirements.txt --no-header --no-annotate

• Check the version of the dependency in the upgraded file.

# upgraded_requirements.txt
(...)
urllib3==1.25.11

• That should be the latest_resolvable_version.

[coretl]

In case this is useful, here is another minimal reproducer: https://github.com/coretl/test_dependabot

It appears that dependabot is updating everything to the latest version, rather than updating to the latest consistent set

[ssbarnea]

I can confirm this bug, I seen it multiple times, last time was on https://github.com/ansible/event-driven-ansible/pull/292/files where it did try to update docutils to a version that was not allowed by other dependencies.

The conflict is caused by:
    antsibull-docs 2.13.1 depends on docutils
    antsibull-changelog 0.29.0 depends on docutils
    sphinx 7.4.7 depends on docutils<0.22 and >=0.20
    sphinx-rtd-theme 2.0.0 depends on docutils<0.21
    The user requested (constraint) docutils==0.21.2

In the past I used two temporary and dangerous workarounds such:

• add problematic dependency to the exclude list for dependabot, can prevent valid bumps.
• add extra conditions inside my own requirements.txt files in order to convince dependabot to avoid doing the invalid bump, again can prevent valid bumps.

The funny bit is that I never seen pip-compile producing invalid updates of dependencies like dependabot. Clearly there is no test of installing the proposed file before the PR is created.