Cannot install with poetry

[stenczelt]

The package cannot be installed with poetry using git+... due to git-only dependencies.

When running

poetry add git+https://github.com/libAtoms/workflow.git

I am getting

Updating dependencies
Resolving dependencies... (4.6s)

Package operations: 23 installs, 0 updates, 0 removals

  • Installing numpy (1.26.2)
  • Installing six (1.16.0)
  • Installing contourpy (1.2.0)
  • Installing cycler (0.12.1)
  • Installing fonttools (4.47.0)
  • Installing kiwisolver (1.4.5)
  • Installing packaging (23.2)
  • Installing pillow (10.1.0)
  • Installing pyparsing (3.1.1)
  • Installing python-dateutil (2.8.2)
  • Installing pytz (2023.3.post1)
  • Installing tzdata (2023.3)
  • Installing click (8.1.7)
  • Installing matplotlib (3.8.2)
  • Installing pandas (2.1.4)
  • Installing scipy (1.11.4)
  • Installing ase (3.22.1)
  • Installing docstring-parser (0.15)
  • Installing expyre-wfl (0.1.1 https://github.com/libAtoms/ExPyRe/tarball/main): Failed

  RuntimeError

  Package https://github.com/libAtoms/ExPyRe/tarball/main cannot be installed in the current environment {'implementation_name': 'cpython', 'implementation_version': '3.11.6', 'os_name': 'posix', 'platform_machine': 'arm64', 'platform_release': '23.2.0', 'platform_system': 'Darwin', 'platform_version': 'Darwin Kernel Version 23.2.0: Wed Nov 15 21:55:06 PST 2023; root:xnu-10002.61.3~2/RELEASE_ARM64_T6020', 'python_full_version': '3.11.6', 'platform_python_implementation': 'CPython', 'python_version': '3.11', 'sys_platform': 'darwin', 'version_info': [3, 11, 6, 'final', 0], 'interpreter_name': 'cp', 'interpreter_version': '3_11'}

  at /opt/homebrew/Cellar/poetry/1.7.1/libexec/lib/python3.12/site-packages/poetry/installation/executor.py:769 in _download_link
      765│             # Since we previously downloaded an archive, we now should have
      766│             # something cached that we can use here. The only case in which
      767│             # archive is None is if the original archive is not valid for the
      768│             # current environment.
    → 769│             raise RuntimeError(
      770│                 f"Package {link.url} cannot be installed in the current environment"
      771│                 f" {self._env.marker_env}"
      772│             )
      773│

Cannot install expyre-wfl.

  • Installing pyyaml (6.0.1)
  • Installing spglib (2.2.0)
  • Installing universalsoap (0.1.0b0 https://github.com/libAtoms/universalSOAP/tarball/main): Failed

  RuntimeError

  Package https://github.com/libAtoms/universalSOAP/tarball/main cannot be installed in the current environment {'implementation_name': 'cpython', 'implementation_version': '3.11.6', 'os_name': 'posix', 'platform_machine': 'arm64', 'platform_release': '23.2.0', 'platform_system': 'Darwin', 'platform_version': 'Darwin Kernel Version 23.2.0: Wed Nov 15 21:55:06 PST 2023; root:xnu-10002.61.3~2/RELEASE_ARM64_T6020', 'python_full_version': '3.11.6', 'platform_python_implementation': 'CPython', 'python_version': '3.11', 'sys_platform': 'darwin', 'version_info': [3, 11, 6, 'final', 0], 'interpreter_name': 'cp', 'interpreter_version': '3_11'}

  at /opt/homebrew/Cellar/poetry/1.7.1/libexec/lib/python3.12/site-packages/poetry/installation/executor.py:769 in _download_link
      765│             # Since we previously downloaded an archive, we now should have
      766│             # something cached that we can use here. The only case in which
      767│             # archive is None is if the original archive is not valid for the
      768│             # current environment.
    → 769│             raise RuntimeError(
      770│                 f"Package {link.url} cannot be installed in the current environment"
      771│                 f" {self._env.marker_env}"
      772│             )
      773│

Cannot install universalSOAP.

I've tried with Python 3.9 as well, no difference.

How to reproduce

Create a pyproject.toml file with the following minimal content (or use poetry init)

[tool.poetry]
name = "wfl-install"
version = "0.1"
description = ""
authors = []

[tool.poetry.dependencies]
python = "~3.11"

[build-system]
requires = ["poetry-core"]
build-backend = "poetry.core.masonry.api"

[bernstei]

I really have no idea what to do with this. I've never used poetry. Do we need to publish expyre and universal-SOAPS in pypi (in which case we might as well do workflow itself as well), so we don't have to do git+https ?

[edited] we could use https://github.com/marketplace/actions/pypi-publish, which seems easy enough. I guess we could do it only on specific (release) tags, but we'd have to get in the habit of creating them.

[bernstei]

Also, I found this https://python-poetry.org/docs/dependency-specification/#git-dependencies

[stenczelt]

Well, publishing on pypi would surely make things simpler and I can highly support that! My suspicion is that the setup.py's definitions of dependencies are not understood by poetry: https://github.com/libAtoms/workflow/blob/69bbae2f7bc01aa59327016f1ba6cbfe42c9f2e5/setup.py#L7-L9

[bernstei]

If we can get someone to adopt the exporting to pypi task, I can help, but don't have time to do it myself. We can discuss on the slack.

For now, do you want to try replacing the the https://github.com/libAtoms/ExPyRe/tarball/main with git+https://github.com/libAtoms/ExPyRe.git, and so on for universalSOAP? Maybe it'd be happier with that syntax.

[bernstei]

@stenczelt do you want to look at this again? Some combination of pypi and going to pyproject.toml might have fixed it.

[stenczelt]

I've tried it again now, and it's working. Thanks!