Support dependency installations from wheel URLs

[fleimgruber]

This is related to https://stackoverflow.com/a/77464614 where a concrete dependency is specified via a requirements.txt file.

Can rye install a concrete dependency such as:

gdal @ https://github.com/cgohlke/geospatial-wheels/releases/download/v2024.2.18/GDAL-3.8.4-cp310-cp310-win_amd64.whl; python_version == "3.10"; sys_platform == "windows"

today or is this planned to be supported?

[mitsuhiko]

The issue with all that stuff is that there is just no standard for how this can be installed other than throwing random args at an installer. I definitely want to find a good answer to this. Today you should be able to add it as a development dependency but that's not a great strategy long term.

[fleimgruber]

Thanks for explaining and considering. "development dependency" as in https://rye-up.com/guide/deps/#development-dependencies? Or do you mean installing it via rye add --dev gdal --path <path-to-downloaded-wheel>?

[rachtsingh]

You should be able to install it if you change that line to:

gdal @ git+https://github.com/cgohlke/geospatial-wheels/releases/download/v2024.2.18/GDAL-3.8.4-cp310-cp310-win_amd64.whl#gdal

though as Armin mentioned it's not yet supported (this syntax is supported by uv, though). I dropped the Python version + Windows mention, by the way (it shouldn't cause any issues since it's in the wheel link already).

[fleimgruber]

@rachtsingh thanks for the heads up. I changed the line in requirements.txt as suggested, but it does not get picked up by rye sync, i.e. it still tries to build from sources (and fails).

I then tried

rye add gdal --dev --git=https://github.com/cgohlke/geospatial-wheels/releases/download/v2024.2.18/GDAL-3.8.4-cp310-cp310-win_amd64.whl#gdal

which errors at rye sync with:

Reusing already existing virtualenv
Generating production lockfile: C:\Users\LeimgruberF\dev\energieatlas-import\requirements.lock
Traceback (most recent call last):
  File "C:\Users\LeimgruberF\.rye\py\cpython@3.10.6\install\lib\runpy.py", line 196, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "C:\Users\LeimgruberF\.rye\py\cpython@3.10.6\install\lib\runpy.py", line 86, in _run_code
    exec(code, run_globals)
  File "C:\Users\LeimgruberF\.rye\pip-tools\cpython@3.10\Scripts\pip-compile.exe\__main__.py", line 7, in <module>
  File "C:\Users\LeimgruberF\.rye\pip-tools\cpython@3.10\lib\site-packages\click\core.py", line 1157, in __call__
    return self.main(*args, **kwargs)
  File "C:\Users\LeimgruberF\.rye\pip-tools\cpython@3.10\lib\site-packages\click\core.py", line 1078, in main
    rv = self.invoke(ctx)
  File "C:\Users\LeimgruberF\.rye\pip-tools\cpython@3.10\lib\site-packages\click\core.py", line 1434, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "C:\Users\LeimgruberF\.rye\pip-tools\cpython@3.10\lib\site-packages\click\core.py", line 783, in invoke
    return __callback(*args, **kwargs)
  File "C:\Users\LeimgruberF\.rye\pip-tools\cpython@3.10\lib\site-packages\click\decorators.py", line 33, in new_func
    return f(get_current_context(), *args, **kwargs)
  File "C:\Users\LeimgruberF\.rye\pip-tools\cpython@3.10\lib\site-packages\piptools\scripts\compile.py", line 659, in cli
    results = resolver.resolve(max_rounds=max_rounds)
  File "C:\Users\LeimgruberF\.rye\pip-tools\cpython@3.10\lib\site-packages\piptools\resolver.py", line 604, in resolve
    is_resolved = self._do_resolve(
  File "C:\Users\LeimgruberF\.rye\pip-tools\cpython@3.10\lib\site-packages\piptools\resolver.py", line 636, in _do_resolve
    resolver.resolve(
  File "C:\Users\LeimgruberF\.rye\pip-tools\cpython@3.10\lib\site-packages\pip\_internal\resolution\resolvelib\resolver.py", line 76, in resolve
    collected = self.factory.collect_root_requirements(root_reqs)
  File "C:\Users\LeimgruberF\.rye\pip-tools\cpython@3.10\lib\site-packages\pip\_internal\resolution\resolvelib\factory.py", line 534, in collect_root_requirements
    reqs = list(
  File "C:\Users\LeimgruberF\.rye\pip-tools\cpython@3.10\lib\site-packages\pip\_internal\resolution\resolvelib\factory.py", line 490, in _make_requirements_from_install_req
    cand = self._make_base_candidate_from_link(
  File "C:\Users\LeimgruberF\.rye\pip-tools\cpython@3.10\lib\site-packages\pip\_internal\resolution\resolvelib\factory.py", line 228, in _make_base_candidate_from_link
    self._link_candidate_cache[link] = LinkCandidate(
  File "C:\Users\LeimgruberF\.rye\pip-tools\cpython@3.10\lib\site-packages\pip\_internal\resolution\resolvelib\candidates.py", line 293, in __init__
    super().__init__(
  File "C:\Users\LeimgruberF\.rye\pip-tools\cpython@3.10\lib\site-packages\pip\_internal\resolution\resolvelib\candidates.py", line 156, in __init__
    self.dist = self._prepare()
  File "C:\Users\LeimgruberF\.rye\pip-tools\cpython@3.10\lib\site-packages\pip\_internal\resolution\resolvelib\candidates.py", line 225, in _prepare
    dist = self._prepare_distribution()
  File "C:\Users\LeimgruberF\.rye\pip-tools\cpython@3.10\lib\site-packages\pip\_internal\resolution\resolvelib\candidates.py", line 304, in _prepare_distribution
    return preparer.prepare_linked_requirement(self._ireq, parallel_builds=True)
  File "C:\Users\LeimgruberF\.rye\pip-tools\cpython@3.10\lib\site-packages\pip\_internal\operations\prepare.py", line 525, in prepare_linked_requirement
    return self._prepare_linked_requirement(req, parallel_builds)
  File "C:\Users\LeimgruberF\.rye\pip-tools\cpython@3.10\lib\site-packages\pip\_internal\operations\prepare.py", line 596, in _prepare_linked_requirement
    local_file = unpack_url(
  File "C:\Users\LeimgruberF\.rye\pip-tools\cpython@3.10\lib\site-packages\pip\_internal\operations\prepare.py", line 157, in unpack_url
    unpack_vcs_link(link, location, verbosity=verbosity)
  File "C:\Users\LeimgruberF\.rye\pip-tools\cpython@3.10\lib\site-packages\pip\_internal\operations\prepare.py", line 80, in unpack_vcs_link
    vcs_backend.unpack(location, url=hide_url(link.url), verbosity=verbosity)
  File "C:\Users\LeimgruberF\.rye\pip-tools\cpython@3.10\lib\site-packages\pip\_internal\vcs\versioncontrol.py", line 606, in unpack
    if os.path.exists(location):
  File "C:\Users\LeimgruberF\.rye\py\cpython@3.10.6\install\lib\genericpath.py", line 19, in exists
    os.stat(path)
TypeError: stat: path should be string, bytes, os.PathLike or integer, not NoneType
error: could not write production lockfile for project

Caused by:
    failed to generate lockfile

I guess I will just wait until this is supported.

[griff-rees]

This may be related: https://github.com/astral-sh/rye/issues/836