Closed InLaw closed 4 years ago
Thanks for that! You where hitting a bug in printing information about the resolution error. It's fixed now in the dev branch.
Your expression calls mach-nix.buildPythonPackage
to build deeplabcut
from github but the requirements you put in are not from deeplabcat. Therefore I assume you just copied the expression head from somewhere else and you actually want to build a python environment, not a python package.
I fixed that and tried to run your expression through mach-nix
The error messages I was getting revealed that your requirements contain a lot of conflicts. I guess you used some exporter to retrieve them. (Pip might accept this since pip just ignores conflicts)
I fixed those conflicts one by one, to see if we're hitting any other bugs and found that mach-nix cannot deal witch python packages which have a dot in their name. I fixed this also on the dev branch. Now your environment is working
Fixing the conflicts requires the following changes yo your requirements:
< dask==2.22.0
---
> dask[complete]==2.22.0
78c78
< llvmlite==0.34.0rc1
---
>
103c103
< osmpythontools==0.2.8
---
> osmpythontools
127c127
< pyct==0.4.6
---
> pyct[cmd]==0.4.6
149c149
< rasterio==1.1.5
---
> rasterio[s3]==1.1.5
The resulting working nix expression is: (also see the provider config at the bottom)
let
mach-nix = import (builtins.fetchGit {
url = "https://github.com/DavHau/mach-nix/";
ref = "dev";
rev = "371c0ebf7d6629c6893bc140be541452b2bd3129";
});
in mach-nix.mkPython {
requirements = ''
affine==2.3.0
appdirs==1.4.4
#appnope==0.1.0; python_version >= "3.3" and sys_platform == "darwin" or platform_system == "Darwin"
arcgis==1.8.2
area==1.1.1
argon2-cffi==20.1.0
#atomicwrites==1.4.0; sys_platform == "win32"
attrs==19.3.0
autoflake==1.3.1
backcall==0.2.0
beautifulsoup4==4.9.1
black==19.10b0
bleach==3.1.5
bokeh==2.1.1
certifi==2020.6.20
cffi==1.14.1
chardet==3.0.4
click==7.1.2
click-plugins==1.1.1
cligj==0.5.0
#colorama==0.4.3; python_version >= "3.3" and sys_platform == "win32" or sys_platform == "win32"
colorcet==2.0.2
cryptography==3.0
cycler==0.10.0
dask[complete]==2.22.0
datashader==0.10.0
datashape==0.5.2
decorator==4.4.2
defusedxml==0.6.0
descartes==1.1.0
entrypoints==0.3
feather-format==0.4.1
fiona==1.8.13.post1
flake8==3.8.3
flufl.enum==4.1.1
geoalchemy2==0.8.4
geofeather==0.3.0
geographiclib==1.50
geojson==2.5.0
geojsonio==0.0.3
geolinks==0.2.0
geopandas==0.8.1
geopy==2.0.0
github3.py==1.3.0
googledrivedownloader==0.4
greedy==0.1.1
idna==2.10
imageio==2.9.0
importlib-metadata==1.7.0
ipykernel==5.3.4
ipython==7.17.0
ipython-genutils==0.2.0
ipywidgets==7.5.1
jedi==0.17.2
jeepney==0.4.3
jinja2==2.11.2
joblib==0.16.0
json5==0.9.5
jsonschema==3.2.0
jupyter-client==6.1.6
jupyter-core==4.6.3
jupyterlab==2.2.4
jupyterlab-server==1.2.0
jwcrypto==0.7
keyring==21.3.0
kiwisolver==1.2.0
lerc==0.1.0
libpysal==4.3.0
lidar==0.5.0
lxml==4.5.2
markupsafe==1.1.1
matplotlib==3.3.0
mccabe==0.6.1
mercantile==1.1.5
missingno==0.4.2
mistune==0.8.4
momepy==0.3.0
more-itertools==8.4.0
multipledispatch==0.6.0
munch==2.5.0
mypy==0.782
mypy-extensions==0.4.3
nbconvert==5.6.1
nbformat==5.0.7
networkx==2.4
notebook==6.1.1
ntlm-auth==1.5.0
numba==0.50.1
numpy==1.19.1
oauthlib==3.1.0
osm-diff-tool==1.1.0
osmapi==1.2.2
osmnet==0.1.6
osmpythontools
overpass==0.7
overpy2==0.4.2
owslib==0.20.0
packaging==20.4
pandas==1.1.0
pandocfilters==1.4.2
param==1.9.3
parso==0.7.1
pathlib==1.0.1
pathspec==0.8.0
pexpect==4.8.0
pickleshare==0.7.5
pillow==7.2.0
pluggy==0.13.1
pooch==1.1.1
prometheus-client==0.8.0
prompt-toolkit==3.0.5
ptyprocess==0.6.0
py==1.9.0
pyarrow==1.0.0
pycodestyle==2.6.0
#pycparser==2.20; platform_system == "Windows" and python_version != "3.3" or platform_system == "Windows" and python_version != "3.3" and sys_platform == "linux"
pycsw==2.4.2
pyct[cmd]==0.4.6
pydal==20200714.1
pyflakes==2.2.0
pygeos==0.7.1
pygis==0.1.3
pygments==2.6.1
#pykerberos==1.2.1; platform_system == "Windows" and sys_platform != "win32"
pyparsing==2.4.7
pyproj==2.6.1.post1
pyrsistent==0.16.0
pyshp==2.1.0
pysimplegui==4.28.0
pytest==5.4.3
python-dateutil==2.8.1
pytz==2020.1
pywavelets==1.1.1
#pywin32==228; sys_platform == "win32" or platform_system == "Windows"
#pywin32-ctypes==0.2.0; sys_platform == "win32"
#pywinpty==0.5.7; os_name == "nt"
pywps==4.2.7
pyyaml==5.3.1
pyzmq==19.0.2
rasterio[s3]==1.1.5
rasterstats==0.15.0
regex==2020.7.14
requests==2.24.0
#requests-kerberos==0.12.0; platform_system == "Windows"
requests-ntlm==1.1.0
requests-oauthlib==1.3.0
requests-toolbelt==0.9.1
richdem==0.3.4
rio-cogeo==1.1.10
rsgis==0.0.3
scikit-image==0.17.2
scikit-learn==0.23.2
scipy==1.5.2
seaborn==0.10.1
secretstorage==3.1.2
send2trash==1.5.0
shapely==1.7.0
simplejson==3.17.2
six==1.15.0
snuggs==1.4.7
soupsieve==1.9.6
spatial==0.2.0
sqlalchemy==1.3.18
supermercado==0.1.1
terminado==0.8.3
testpath==0.4.4
threadpoolctl==2.1.0
tifffile==2020.7.24
toml==0.10.1
toolz==0.10.0
tornado==6.0.4
tqdm==4.48.2
traitlets==4.3.3
typed-ast==1.4.1
typing-extensions==3.7.4.2
ujson==3.1.0
uritemplate==3.0.1
urllib3==1.25.10
verde==1.5.0
wcwidth==0.2.5
webencodings==0.5.1
werkzeug==1.0.1
widgetsnbextension==3.5.1
#winkerberos==0.7.0; platform_system == "Windows" and sys_platform == "win32" or platform_system == "Windows"
xarray==0.16.0
xarray-spatial==0.0.8
xmltodict==0.12.0
zipp==3.1.0
'';
providers = {
# Change the provider for osmpythontools to "nixpkgs", because the inherited patches from nixpkgs won't apply.
osmpythontools = "nixpkgs";
};
}
eg. rasterio==1.1.5 > rasterio[s3]==1.1.5 and to choose the best provider for osmpythontools
it shows:
Multiple nixkgs attributes found for tornado-6.0.4: ['tornado_4', 'tornado']
Picking 'tornado' as base attribute name.
why is tornado installed as "tornado - 6.0.4 - sdist" if
providers = {
_default = "nixpkgs,wheel,sdist";
}
[x] there should be a "python3.7-tornado-6.0.4 | python37Packages.tornado"
[x] why is there no wheel for it?
How could I debug the changes by myself? eg. rasterio==1.1.5 > rasterio[s3]==1.1.5 and to choose the best provider for osmpythontools
You can either install the rasterio package with s3 or without s3 but you cannot have both variants at the same time, therefore the resolver raises a conflict.
Therefore if your top level requirements contain rasterio==1.1.5
and a conflict appears between rasterio and rasterio[s3], you know that one of your subdependencies must have declared rasterio[s3]
. This can either be fixed by adapting your top level dependencies or removing the sub dependency which requires rasterio[s3]
.
and to choose the best provider for osmpythontools
My train of thought here:
sdist
( -> means no wheel available)sdist
provider inherit patches from nixpkgs. In this case they probably don't fit with the current version of osmpythontools.Now I can try one of two things:
overrides_pre
to set patches = []
Multiple nixkgs attributes found for tornado-6.0.4: ['tornado_4', 'tornado'] Picking 'tornado' as base attribute name. what does it mean?
For different reasons, like inheriting patches or excluding the chance of collisions, mach-nix must know about all candidates for a package in nixpkgs. Also if you're not using the nixpkgs
provider. It will always try to override the candidate which seems most similar (closest version) than the one which is built. If it cannot decide, because the versions are too far apart, it shows this message and just picks one of both. Maybe it's confusing and I should hide it since it doesn't really matter in most cases.
there should be a "python3.7-tornado-6.0.4 | python37Packages.tornado"
Tornado only releases wheels for windows
is it possible to activate a cache for wheel packages?
Why would you want to do that? Wheel packages are very quick to install. If they contain extension modules, they are already in binary format and just need to be patched. There wouldn't be much gain in caching them. Caching sdist
packages would be more of a benefit.
From my point of view, it is always good to get those information (so that you are aware of what is happening)
error: