search

Materials-Data-Science-and-Informatics / fair-python-cookiecutter

An opinionated cookiecutter template to kickstart a modern best-practice Python project with FAIR metadata.
MIT License
27 stars 4 forks source link

installation instructions not work #14

Closed daniel-mohr closed 1 year ago

daniel-mohr commented 1 year ago

Describe the bug

The proposed installation instruction does not work.

To Reproduce Steps to reproduce the behavior:

  1. start ubuntu:20.04 or ubuntu:22.04 in docker
  2. apt-get update
  3. apt-get install python3 python3-pip git
  4. pip install git+ssh://git@github.com:Materials-Data-Science-and-Informatics/fair-python-cookiecutter-demo.git

leads to:

...
  ssh: Could not resolve hostname github.com:Materials-Data-Science-and-Informatics: Name or service not known
  fatal: Could not read from remote repository.
...

Solutions? For ubuntu:22.04 it could be fixed by using https: pip install git+https://github.com/Materials-Data-Science-and-Informatics/fair-python-cookiecutter-demo.git

I'm not sure if ubuntu:20.04 in your scope. But this does not work at all:

root@ae7c5ae64430:/# pip install git+https://github.com/Materials-Data-Science-and-Informatics/fair-python-cookiecutter-demo.git
Collecting git+https://github.com/Materials-Data-Science-and-Informatics/fair-python-cookiecutter-demo.git
  Cloning https://github.com/Materials-Data-Science-and-Informatics/fair-python-cookiecutter-demo.git to /tmp/pip-req-build-pzr9njhg
  Running command git clone -q https://github.com/Materials-Data-Science-and-Informatics/fair-python-cookiecutter-demo.git /tmp/pip-req-build-pzr9njhg
ERROR: Exception:
Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/pip/_internal/cli/base_command.py", line 186, in _main
    status = self.run(options, args)
  File "/usr/lib/python3/dist-packages/pip/_internal/commands/install.py", line 357, in run
    resolver.resolve(requirement_set)
  File "/usr/lib/python3/dist-packages/pip/_internal/legacy_resolve.py", line 177, in resolve
    discovered_reqs.extend(self._resolve_one(requirement_set, req))
  File "/usr/lib/python3/dist-packages/pip/_internal/legacy_resolve.py", line 333, in _resolve_one
    abstract_dist = self._get_abstract_dist_for(req_to_install)
  File "/usr/lib/python3/dist-packages/pip/_internal/legacy_resolve.py", line 282, in _get_abstract_dist_for
    abstract_dist = self.preparer.prepare_linked_requirement(req)
  File "/usr/lib/python3/dist-packages/pip/_internal/operations/prepare.py", line 515, in prepare_linked_requirement
    abstract_dist = _get_prepared_distribution(
  File "/usr/lib/python3/dist-packages/pip/_internal/operations/prepare.py", line 95, in _get_prepared_distribution
    abstract_dist.prepare_distribution_metadata(finder, build_isolation)
  File "/usr/lib/python3/dist-packages/pip/_internal/distributions/sdist.py", line 33, in prepare_distribution_metadata
    self.req.load_pyproject_toml()
  File "/usr/lib/python3/dist-packages/pip/_internal/req/req_install.py", line 512, in load_pyproject_toml
    pyproject_toml_data = load_pyproject_toml(
  File "/usr/lib/python3/dist-packages/pip/_internal/pyproject.py", line 75, in load_pyproject_toml
    pp_toml = toml.load(f)
  File "/usr/share/python-wheels/toml-0.10.0-py2.py3-none-any.whl/toml/decoder.py", line 134, in load
    return loads(f.read(), _dict, decoder)
  File "/usr/share/python-wheels/toml-0.10.0-py2.py3-none-any.whl/toml/decoder.py", line 455, in loads
    ret = decoder.load_line(line, currentlevel, multikey,
  File "/usr/share/python-wheels/toml-0.10.0-py2.py3-none-any.whl/toml/decoder.py", line 725, in load_line
    value, vtype = self.load_value(pair[1], strictly_valid)
  File "/usr/share/python-wheels/toml-0.10.0-py2.py3-none-any.whl/toml/decoder.py", line 802, in load_value
    return (self.load_array(v), "array")
  File "/usr/share/python-wheels/toml-0.10.0-py2.py3-none-any.whl/toml/decoder.py", line 914, in load_array
    a[b] = a[b] + ',' + a[b + 1]
IndexError: list index out of range
apirogov commented 1 year ago

Thanks for your report!

  1. Are you using the correct pip? System-level pip in ubuntu can be to Python 2.7, so possibly you have to use pip3
  2. Could you say which version of pip it is (and also of Python interpreter)? maybe it does not support the syntax?

Just in case there was some confusion:

But it looks like you were trying to install the demo repo, which does not contain anything useful at all, and in a real-world setting probably one of the things the user should do is enabling PyPI releases if it is a tool and adapt the documentation accordingly.

To instantiate the repo you have to run cookiecutter https://github.com/Materials-Data-Science-and-Informatics/fair-python-cookiecutter.

(In any case, of course installing the resulting Python package is supposed to work, so the issue is absolutely valid)

daniel-mohr commented 1 year ago

I'm really mean here the demo repo -- it is more or less the output of your cookiecutter.

I run my short tests in a docker shell, e. g. docker run --mount type=bind,source=$(pwd),destination=/app --rm -it ubuntu:22.04.

Regarding pip: I installed only python3-pip therefore it is really the python3 version:

ubuntu:22.04:

# pip --version
pip 22.0.2 from /usr/lib/python3/dist-packages/pip (python 3.10)

ubuntu:20.04:

# pip --version
pip 20.0.2 from /usr/lib/python3/dist-packages/pip (python 3.8)
apirogov commented 1 year ago

@daniel-mohr so there are two issues here actually

URL does not work with pip in the given docker setup

I can reproduce your problem with the URL resolution inside docker, yet with the same pip version (22.0.2) works outside of docker. It looks like an issue that is not related to the template, but rather your way of testing with the docker shell. The used URL format is supported by pip for a long time, see e.g. here, so maybe your issue is because the docker shell has not enough networking setup properly / DNS resolution failure.

pip fails to build package

Concerning the other issue - I've tested with different pip versions and apparently it does not work with pip < 21.2, because of a bug in the toml parsing in pip, i.e. creating problems with the pip version provided in Ubuntu 20.04.

the old pip seems to choke on this, which is valid TOML 1.0:

include = [
  "*.md", "LICENSE", "LICENSES", ".reuse/dep5", "CITATION.cff", "codemeta.json",
  { path = "mkdocs.yml", format = "sdist" },
  { path = "docs", format = "sdist" },
  { path = "tests", format = "sdist" },
]

I've already stumbled on this problem in various contexts where Python projects still used the broken toml package instead of the tomli / tomllib, and I won't butcher the project configuration to accomodate an old broken parser (here I just want to follow best practice and not bundle docs in the wheels distribution, but include them in the source dist).

So the workaround is to simply update to a recent version of pip (pip install --upgrade pip) before trying to install the package.

I fear in this case there's no other solution, because whatever pip is currently installed must process the pyproject.toml first, even to know that it needs to update itself. I might add a note somewhere in the README/docs.