Closed rocketinventor closed 3 years ago
The project uses poetry for dependencies management, requirements.txt
is actually generated from the poetry settings: https://python-poetry.org/docs/cli/#export
I think it would make more sense to make these changes on the pyproject.toml
and keep one source of truth for the dependencies version
Well, I thought that it was an issue from dependabot, but I checked now, and I can see that it only updated the .lockfile.
If the requirements.txt file is indeed being managed by Poetry, then I am not sure how to fix this issue...
Poetry doesn't seem to give any options about how the requirements.txt file is generated. Lxml and certifi are not specified directly as a requirement, but they are sub-dependencies. The only dependencies listed on the pyproject.toml file are:
chromedriver-autoinstaller = "^0.2.2" colorama = "^0.4.3" EbookLib = "^0.17.1" requests = "^2.24.0" selenium = "^3.141.0" selenium-wire = "^2.1.0"
Therefore, I would be unsure of how to fix this issue with Poetry (other than to manually edit the file).
Please feel free to advise, or update any settings that might fix this.
@leoncvlt I added in a script to parse the pyproject.toml
file and made a commit with the new generated requirements.txt
file.
Also, I told Git to ignore the .lock
file, because that really is meant to stay on one person's computer, only. This will have the side-benefit that dependabot will not have to bother us every time a sub-dependency is updated.
All that needs to be done to update the dependencies is to run py pyproject_parse.py
in the project folder before committing. A nice and clean requirements.txt
file that matches the pyproject.toml
file will be automatically generated.
These changes should accommodate your goal to "keep one source of truth for the dependencies version". I think that this pull-request can be merged now.
Thanks @rocketinventor, this is a good way to go!
Updated: Old text has been italicized and
crossed out.The version number listed for
lxml
is not the newest one available (4.6.2
for Python 3.9). The "==" operator forces pip to uselxml
version4.5.2
, even if that meant downgrading it. This causes errors forlxml
if "Microsoft Visual C++ 14.0" is not installed (because it is needed for compiling).Also, some packages like
certifi
always need to be up to date (for securities' sake). They can now be updated as desired, and will not have to be rolled back in order to installblinkist-scraper
.Furthermore, the package versions and requirements lists specified by
pyproject.toml
,poetry.lock
, andrequirements.txt
are all different, creating fragmentation in regards to what is actually installed when following the instructions.Since it doesn't seem like any of these packages require a specific version, I have changed all of them to minimum version numbers (instead of required ones) using the>=
operator.This pull request updates the
requirements.txt
to exactly match the requirements specified bypyproject.toml
and adds in a new tool to keep them in sync. Also, it ditches thepoetry.lock
file which was causing conflict about what actually needs to be installed. All the requirement specifications can now be kept in loc-step (instead of different specifications in different files, as was the case before). Pip already knows how to install dependencies properly. Now it can do so without having to have its hands tied. No C++ compiler needed!**anymore
Closes #28