Open msftcangoblowm opened 2 days ago
These looks at the venv, not the requirements files:
pip list --not-required
pip list --outdated
pip check
These are very useful for managing venv.
For the requirements files, would like similar tools, preferably automated. Should help decide which packages' version is needed and update the appropriate .in
requirements files.
pins-\*.in
(issue specific nudge pins) filesOn difficult issues, an issue specific nudge pins file is needed. But for mere package version disparity issues, a single nudge pins file is proposed.
This gets recompiled before .unlock
files cuz these files are ideal to check whether package version conflicts exists.
This is a resolved, not locked file. After nudge pin files are updated, to reflect the changes, .unlock
file would need to be recompiled
pins-cffi.in
# musllinux aarch64 support; 3.13 support; fixes distutils.msvc9compiler failing imports
cffi>=1.17.1
This issue specific nudge pin file says the reason is:
added support for py313
musllinux aarch64 support
fixes distutils.msvc9compiler failing imports
The gh workflows for testing, the strategy.matrix include py313 and Windows OS.
So this package version is needed. There is little need to ever remove this nudge pin. Possibly manually adjust as a hypothetical future release adds support or fixes important issues.
This nudge pin is a good thing. Shows cffi team are doing a beyond everyones' expectations incredible job.
alias dgrep='function _dgrep(){ cd "$1" 2>/dev/null && grep -rInH "$2" $1; };_dgrep'
cd requirements
dgrep . "pins-cffi.in"
./manage.in:7:-c pins-cffi.in ./dev.in:5:-c pins-cffi.in
dgrep . "cffi=="
./manage.lock:11:cffi==1.17.1
./dev.lock:17:cffi==1.17.1
Luckily cffi package is in sync
Without the ==
dgrep . "cffi"
./manage.unlock:19:cffi>=1.17.1 ./pins-cffi.in:2:cffi>=1.17.1 ./manage.in:7:-c pins-cffi.in ./manage.lock:11:cffi==1.17.1 ./manage.lock:95: # via cffi ./dev.lock:17:cffi==1.17.1 ./dev.lock:118: # via cffi ./dev.unlock:26:cffi>=1.17.1 ./dev.in:5:-c pins-cffi.in
docs/
folder has it's own venv. cffi is not a dependency. This is the reason why pins-cffi.in
is not a pins-cffi.shared.in
. The cffi package is not shared requirement in the same way prod.shared.in
or pins.shared.in
are.
Issue description
Each requirement file is created by a call to
pip-compile
. Long story short, the package versions are not sync'ed after multiple pip-compile calls.grep .lock files in requirements/*.lock, for a specific package, will reveal packages with different required versions. One venv can only hold one package version resulting in as predictable as rain in autumn dependency resolution conflicts.
.in
requirements file can have both constraints-c
and included-r
requirements:For an app, the packages dependencies are normally locked.
prod.shared.lock
would be set in pyproject.toml.This is an unusual situation. And the dev might choose to keep the production venv unlocked.
For normal situation, packages are unlocked.
prod.shared.unlock
would be set in pyproject.toml.Shared means this constraints file will be included across multiple venv. Extreme care must be taken. Any nudge pins would be general in nature, e.g. tomli or typing_extensions
requirements/pins-cffi.in
Gives the opportunity to document why there is a package version preference (nudge pin). Upstream issues urls to track the issue and ongoing work. With this info everyone, especially the original dev, have a chance to understand both cause and source. Then can decide if the nudge pin is still warranted.
The normal status quo is nudge pins have no accompanying documentation. When issues come up concerning a nudge pin, the background has to to be tracked down. If it even exists. With nudge pin documentation, all background info would be in one place.
Occurs when
After updating requirements files via multiple
pip-compile
calls.always occurs and is very time consuming to get right manually
Would Resolve
Remove the need to check the versions of every single package
Proposed fix/feature
Need a process which occurs after the multiple
pip-compile
calls, which syncs the packages in the requirements files