packaging with uv+hatchling and then pip install --editable crashes with: patchspec has no attribute GitIgnoreSpec

kaddkaka commented 2 months ago

Description

I'm trying to package my first python project (so I might be making stupid mistakes) but I have tried in different ways the week. My package is called prawk and I want to be able to do import prawk in a python file outside this package. When I perform pip install --editable . in my project top folder (pyproject.toml) is located, I get error below.

Expected behavior

installation to succeed

pip version

pip 22.0.2 from /usr/lib/python3/dist-packages/pip (python 3.10)

Python version

Python 3.10.12

OS

Ubuntu 22.04

How to Reproduce

My project structure can been at https://github.com/kaddkaka/pawk
Clone or make a similar project. (the pyproject is configured to package with hatchling)
First I installed it with uv using uv tool install .
Then I tried installing with pip install --editable . -> error output

Output

:

Defaulting to user installation because normal site-packages is not writeable
Obtaining file:///home/david/projects/prawk
  Installing build dependencies ... done
  Checking if build backend supports build_editable ... done
  Getting requirements to build editable ... done
  Installing backend dependencies ... done
  Preparing editable metadata (pyproject.toml) ... done
Building wheels for collected packages: prawk
  Building editable for prawk (pyproject.toml) ... error
  error: subprocess-exited-with-error

  × Building editable for prawk (pyproject.toml) did not run successfully.
  │ exit code: 1
  ╰─> [26 lines of output]
      Traceback (most recent call last):
        File "/usr/lib/python3/dist-packages/pip/_vendor/pep517/in_process/_in_process.py", line 363, in <module>
          main()
        File "/usr/lib/python3/dist-packages/pip/_vendor/pep517/in_process/_in_process.py", line 345, in main
          json_out['return_val'] = hook(**hook_input['kwargs'])
        File "/usr/lib/python3/dist-packages/pip/_vendor/pep517/in_process/_in_process.py", line 283, in build_editable
          return hook(wheel_directory, config_settings, metadata_directory)
        File "/tmp/pip-build-env-e8xx31z6/overlay/local/lib/python3.10/dist-packages/hatchling/build.py", line 83, in build_editable
          return os.path.basename(next(builder.build(directory=wheel_directory, versions=['editable'])))
        File "/tmp/pip-build-env-e8xx31z6/overlay/local/lib/python3.10/dist-packages/hatchling/builders/plugin/interface.py", line 155, in build
          artifact = version_api[version](directory, **build_data)
        File "/tmp/pip-build-env-e8xx31z6/overlay/local/lib/python3.10/dist-packages/hatchling/builders/wheel.py", line 494, in build_editable
          return self.build_editable_detection(directory, **build_data)
        File "/tmp/pip-build-env-e8xx31z6/overlay/local/lib/python3.10/dist-packages/hatchling/builders/wheel.py", line 505, in build_editable_detection
          for included_file in self.recurse_selected_project_files():
        File "/tmp/pip-build-env-e8xx31z6/overlay/local/lib/python3.10/dist-packages/hatchling/builders/plugin/interface.py", line 181, in recurse_selected_project_files
          yield from self.recurse_explicit_files(self.config.only_include)
        File "/tmp/pip-build-env-e8xx31z6/overlay/local/lib/python3.10/dist-packages/hatchling/builders/plugin/interface.py", line 267, in recurse_explicit_files
          if self.config.include_path(relative_file_path, explicit=True, is_package=is_package):
        File "/tmp/pip-build-env-e8xx31z6/overlay/local/lib/python3.10/dist-packages/hatchling/builders/config.py", line 98, in include_path
          and not self.path_is_excluded(relative_path)
        File "/tmp/pip-build-env-e8xx31z6/overlay/local/lib/python3.10/dist-packages/hatchling/builders/config.py", line 113, in path_is_excluded
          if self.exclude_spec is None:
        File "/tmp/pip-build-env-e8xx31z6/overlay/local/lib/python3.10/dist-packages/hatchling/builders/config.py", line 215, in exclude_spec
          self.__exclude_spec = pathspec.GitIgnoreSpec.from_lines(all_exclude_patterns)
      AttributeError: module 'pathspec' has no attribute 'GitIgnoreSpec'
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
  ERROR: Failed building editable for prawk
Failed to build prawk
ERROR: Could not build wheels for prawk, which is required to install pyproject.toml-based projects

Am I missing something? Not sure what to do from here.

henryiii commented 2 months ago

I wasn't able to reproduce:

uv tool install .
uv venv
. .venv/bin/activate
uv pip install pip
pip install --editable .

Defaulting to user installation because normal site-packages is not writeable

You shouldn't be trying to install anything into a system install, or especially a user install. I'd always use venvs, like you see above. Not the problem here, though.

kaddkaka commented 2 months ago

So if I want something installed in the global environment, available from just invoking python3 from the command line. How would I do that?

kaddkaka commented 2 months ago

I wasn't able to reproduce:
uv tool install .
uv venv
. .venv/bin/activate
uv pip install pip
pip install --editable .
Defaulting to user installation because normal site-packages is not writeable

You shouldn't be trying to install anything into a system install, or especially a user install. I'd always use venvs, like you see above. Not the problem here, though.

I did not do any of these steps:

uv venv
. .venv/bin/activate
uv pip install pip

Are they needed? Why? What do they do? Isnt this creating a venv and activating it inside the packge itself? I want to use the package from outside (other package/global env).

henryiii commented 2 months ago

If you want a command to be globally available, that's what you use uv tool install (or pipx install) for. Those exist entirely to expose your console entry-points without making the package installable. You don't need them otherwise. If you want to be able to import it, you should have a virtual environment. Installing into a single, overall environment can break your system Python, interfere with other packages, be hard to uninstall, etc. If it's a Python you've personally installed, then you can do it (since you can just wipe the install out), but your Python (above) is managed by the system, which is why it's not letting you install to it. Instead, it's installing to a "user" install, which will be shared among all Python instances, which is really fragile and easy to break.

But that's not why... Oh, if you've been using your user install, maybe you have an old pathspec there that's breaking the isolated install? What is the output of pip list --user?

(UV completely disallows the --user location because of how easy it is to break things when using it, by the way)

kaddkaka commented 2 months ago

$ pip list --user | grep pathspec
pathspec              0.11.1

I might have done one or two [sudo] pip install X previously (like matplotlib). I'm just interested to get my package as easily importable for when I run random python scripts or python one-liners directly from command line. Isn't that user or global env? What commands should I run to get my package available as python3 -c 'import my_package'?

Got these python3 executables:

$ which -a python3
/usr/bin/python3
/bin/python3

pypa / hatch