search

jazzband / pip-tools

A set of tools to keep your pinned Python dependencies fresh.
https://pip-tools.rtfd.io
BSD 3-Clause "New" or "Revised" License
7.69k stars 610 forks source link

The `--upgrade` option is not written to requirements.txt #2008

Closed rbuffat closed 5 months ago

rbuffat commented 10 months ago

When I generate a requirements.txt using the following command:

pip-compile --allow-unsafe --strip-extras --upgrade requirements.in

pip-compile generates the following header in requirements.txt:

#
# This file is autogenerated by pip-compile with Python 3.11
# by the following command:
#
#    pip-compile --allow-unsafe --strip-extras requirements.in
#

The --upgrade option is somehow missing. I was wondering if this is the intended behavior and if so why.

I noticed that when we wanted to upgrade Django to a new major version GitHubs dependabot never opened a pull request despite a new LTS release was released for some time. The reason was that pinned dependencies were too old and prevented dependabot to update Django due to dependencies requiring an older version of Django. Deleting requirements.txt and regenerating it solved the issue. I'm not sure if including --upgrade solves that dependabot also updates dependencies, but as far as I remember I read somewhere that dependabot respects the pip-compile options in requirements.txt.

chrysle commented 10 months ago

The --upgrade option is somehow missing. I was wondering if this is the intended behavior and if so why.

That's surely not intended, thanks for reporting!

AndydeCleyre commented 10 months ago

That's surely not intended, thanks for reporting!

Well it is/has-been intended, see #1461 and #1675, though not without surprising some users along the way. You can set CUSTOM_COMPILE_COMMAND to force the header command to include it.

chrysle commented 10 months ago

I was unaware of that, thank you! Maybe add an example and explanation to the paragraph, then?

rbuffat commented 10 months ago

Thanks @AndydeCleyre and @chrysle.

This " one-off option" is really confusing and seems not to be documented. I would also suggest changing the header because the statement that the file is created with the stated command is clearly not true when --upgrade is used. 

#
# This file is autogenerated by pip-compile with Python 3.11
# by the following command:
#
#    pip-compile --allow-unsafe --strip-extras requirements.in
#
WhyNotHugo commented 10 months ago

The same file can be generated with the above command. If you provide --upgrade, it is possible that the generated file will be different.

This is a reference of what command to use when needing to regenerate the file. For example, adding new dependencies: you add it to the source file, and then run this command.

The --upgrade flag is not included in the same way that --upgrade-package Pillow is not included. It would be senseless to include --upgrade-package Pillow; this is a one-off addition to the file generation, not something that you would provide every time.

adamchainz commented 5 months ago

I agree, it doesn’t make sense to include it.

@rbuffat said:

“This " one-off option" is really confusing and seems not to be documented.”

It’s documented in the readme and pip-compile -h says:

  -U, --upgrade / --no-upgrade    Try to upgrade all dependencies to their
                                  latest versions
atugushev commented 5 months ago

Why it's considered one-off?

To make requirement.txt less noisy in VCS when you upgrade dependencies occasionally. For example, --upgrade-pakage="django==3.2.3" option is not expected in the header after pip-compile run, since it's a one-off operation. The same principle applied to --upgrade option for most cases.

Originally posted by @atugushev in https://github.com/jazzband/pip-tools/issues/1461#issuecomment-892242858

atugushev commented 5 months ago

Consider using CUSTOM_COMPILE_COMMAND for any special cases. Thanks for the issue nonetheless. I'll close the issue based on above.