Closed mfisher87 closed 2 weeks ago
My two cents, but not trying to throw a wrench in this if y'all are already all for it...
I really like poetry. I've found it easier for package and version management in other projects. And I've had no issue using pip install
with poetry packages before.
FWIW when I try pip install -e .
with the latest pip=23.3.1
release I get
Running setup.py develop for earthaccess
error: subprocess-exited-with-error
× python setup.py develop did not run successfully.
│ exit code: 1
╰─> [1 lines of output]
ERROR: Can not execute `setup.py` since setuptools is not available in the build environment.
[end of output]
note: This error originates from a subprocess, and is likely not a problem with pip.
error: subprocess-exited-with-error
× python setup.py develop did not run successfully.
│ exit code: 1
╰─> [1 lines of output]
ERROR: Can not execute `setup.py` since setuptools is not available in the build environment.
[end of output]
note: This error originates from a subprocess, and is likely not a problem with pip.
Interesting. I wonder if the below is true IFF setuptools is actually installed?
(NOTE: Modern pip can find dependencies that are specified somewhere other than project.dependencies, so
pip install -e .
does work with new enough version of pip. We think it's looking at[build-system]
section to discover poetry dependencies.)
@jrbourbeau — to clarify — are you trying to use a local, edited, version of a dependency? Something like what's described in this poetry issue?
@danielfromearth we solved @jrbourbeau issue here: https://github.com/nsidc/earthaccess/pull/402#discussion_r1416611230
We needed to update the build backend and remove the dummy setup.py
file as pip has behavioral changes if that file exists or not.
oh, that's great! It's nice to have only pyproject.toml
and not both that and setup.py
I'd suggest considering uv. It might not quite be ready for prime-time, but I strongly feel this is worth a look. It's by the same folks that created ruff, which we are already using here.
As a follow-on, I know some people love poetry and others hate it. It's difficult to find any sort of tool or library that is nearly universally appreciated, but poetry might be in the class of tools where more folks are at the extremes of the spectrum than more centrally located.
For those who are not so much in love with poetry, a major factor is that it can be excruciatingly slow to resolve dependencies (as is the case for some other tools as well, such as pipenv). Here's an example that I just ran into on this project:
(earthaccess-dev) $ poetry add importlib-resources
Using version ^6.3.2 for importlib-resources
Updating dependencies
Resolving dependencies... (134.2s)
...(snip)...
- Updating mkdocs-jupyter (0.22.0 /home/conda/feedstock_root/build_artifacts/mkdocs-jupyter_1663979448056/work -> 0.22.0)
Writing lock file
(earthaccess-dev) 2m55s $
That took almost 3 minutes (134 seconds for resolution alone, plus another ~40 seconds after that) to add a seemly innocuous library that has only 1 transitive dependency (zipp, which has no further dependencies), and that's not even as bad as I've encountered elsewhere, where it can take many 10s of minutes to resolve dependencies.
Along with uv
, rye is another option to consider. Both are implemented in Rust and are blazingly fast at resolving dependencies.
As far as I've looked, rye
appears to be a bit more straightforward than uv
, however, the creator of rye
has joined forces with Astral (creators of uv
), so it's not clear yet how things will evolve (i.e., whether rye
will be subsumed by uv
, or if it will remain a distinct tool), so it may behoove us to keep an eye on their progression in the near future before committing to one or the other (or to some other tool).
Just noting that uv
has --resolution=lowest
, which will install the lowest compatible version of dependencies, which would mean we wouldn't need to maintain a separate minimum-bounds environment as noted in #497 .
https://github.com/astral-sh/uv?tab=readme-ov-file#resolution-strategy
Just noting that uv has --resolution=lowest, which will install the lowest compatible version of dependencies, which would mean we wouldn't need to maintain a separate minimum-bounds environment
:star_struck:
Boosting this issue, which could be said to be blocking #520, because I'm not sure how to approach resolving environments for CI. We need python-cmr>=0.10.0
now, and the current approach gets it from conda-forge (where it does not yet exist). I assume we want to let poetry or uv manage this testing environment rather than separately maintaining a conda environment.yml?
Boosting this issue, which could be said to be blocking #520, because I'm not sure how to approach resolving environments for CI. We need
python-cmr>=0.10.0
now, and the current approach gets it from conda-forge (where it does not yet exist). I assume we want to let poetry or uv manage this testing environment rather than separately maintaining a conda environment.yml?
To get around the build failure for that issue, perhaps just move python-cmr
to a pip
install within ci/environment-mindeps.yaml
. Specifically, replace this line:
- python-cmr=0.9.0
with these lines:
- pip
- pip:
- python-cmr==0.10.0
Open an issue like this one if you'd like to become a maintainer over on the conda-forge feedstock :) It would be really useful to have a couple more folks helping out there I think!
https://github.com/conda-forge/python-cmr-feedstock/issues/7
Kinda related #614
@MattF-NSIDC : Little experience with poetry, prefer setuptools
@jhkennedy : Has #Reasons, happy to elaborate if the group is amenable to switching
@jrbourbeau : "I want to be able to
pip install -e .
" (NOTE: Modernpip
can find dependencies that are specified somewhere other thanproject.dependencies
, sopip install -e .
does work with new enough version ofpip
. We think it's looking at[build-system]
section to discover poetry dependencies.)@betolink : "less is more"