juftin / hatch-pip-compile

hatch plugin to use pip-compile (or uv) to manage project dependencies and lockfiles
http://juftin.com/hatch-pip-compile/
MIT License
74 stars 3 forks source link

Installation instructions not working #53

Closed Tobi-De closed 8 months ago

Tobi-De commented 8 months ago

Hi, I don't know where the issue is from but this code

[tool.hatch.env]
requires = [
    "hatch-pip-compile"
]

does not work, adding it make the pyproject.toml invalid and hatch fails to parse the file with FileNotFoundError: [Errno 2] No such file or directory: '1', I'm on hatch latest version 1.9.1. I also tried

[build-system]
requires = ["hatchling", "hatch-pip-compile"]
build-backend = "hatchling.build"

but it seems to have no effect

juftin commented 8 months ago

Can you share your entire pyproject.toml? Or even better, a link to the GitHub repository.

Tobi-De commented 8 months ago

Can you share your entire pyproject.toml? Or even better, a link to the GitHub repository.

https://github.com/Tobi-De/vagabond

Tobi-De commented 8 months ago

I just tried hatch-pip-compile on a freshly generated hatch project and it worked no issue, now I'm more confused, the issue is definitively on my end, this is the pyproject content

[build-system]
requires = ["hatchling"]
build-backend = "hatchling.build"

[project]
name = "vagabond"
version = "0.1.0"
description = ''
readme = "README.md"
requires-python = ">=3.11"
authors = [{ name = "Tobi DEGNON", email = "tobidegnon@proton.me" }]
dependencies = [
  "Pillow>=10.1.0",
  "hiredis>=2.2.3",
  "redis>=5.0.1",
  "boto3>=1.26.160",
  "gunicorn",
  "sentry-sdk>=1.26.0",
  "whitenoise[brotli]>=6.5.0",
  "psycopg[c]",
  "django[argon2]>=4.2.0,<5.0",
  "django-environ>=0.10.0",
  "django-model-utils>=4.3.1",
  "django-allauth>=0.54.0",
  "django-template-partials>=23.4",
  "django-tailwind-cli>=2.7.2",
  "django-extensions>=3.2.3",
  "django-htmx>=1.15.0",
  "django-anymail>=10.0",
  "django-browser-reload>=1.10.0",
  "django-debug-toolbar>=4.1.0",
  "django-fastdev>=1.7.3",
]

[tool.hatch.env]
requires = [
    "hatch-pip-compile"
]

[tool.hatch.envs.default]
type = "pip-compile"
dependencies = ["dj-notebook>=0.6.1", "Werkzeug[watchdog]>=3.0.1"]

[tool.hatch.envs.default.scripts]
server = "python manage.py tailwind runserver {args}"
migrate = "python manage.py migrate {args}"
makemigrations = "python manage.py makemigrations {args}"
makesuperuser = "python manage.py makesuperuser"
reset-db = "python manage.py reset_db --noinput"
shell = "python manage.py shell_plus {args}"
reset-migrations = ["falco rm-migrations", "makemigrations"]

[[tool.hatch.envs.all.matrix]]
python = ["3.11", "3.12"]

[tool.hatch.envs.test]
dependencies = [
  "coverage[toml]>=6.5",
  "pytest>=7.4.0",
  "pytest-django>=4.5.2",
  "pytest-sugar>=0.9.7",
]

[tool.hatch.envs.test.scripts]
test = "pytest {args:tests}"
test-cov = "coverage run -m pytest {args:tests}"
cov-report = ["- coverage combine", "coverage report"]
cov = ["test-cov", "cov-report"]

[tool.hatch.envs.lint]
dependencies = [
  "pyright>=1.1.340",
  "curlylint>=0.13.1",
  "reorder-python-imports>=3.12.0",
  "djhtml>=3.0.6",
  "ruff>=0.1.8",
  "pyupgrade>=3.15.0",
  "django-upgrade>=1.15.0",
  "autoflake>=2.2.1",
]

[tool.hatch.envs.lint.scripts]
run-pyright = "pyright {args:.}"
run-format = "hatch fmt --formatter"
run-autoflake = "autoflake --in-place --remove-all-unused-imports --remove-unused-variables --expand-star-imports --exclude=.*/__init__.py {args:.}"
run-reorder-python-imports = "reorder-python-imports config/*.py {args:./*/*/*.py}"
run-ruff = "ruff check --quiet --fix --exit-non-zero-on-fix {args:.}"
run-curlylint = "curlylint {args:.}"
run-djhtml = "djhtml --tabwidth 4 {args:.}"
run-pyupgrade = "pyupgrade --py311-plus {args}"
run-all-pyupgrade = "git ls-files -- '*.py' | xargs pyupgrade --py311-plus"
run-django-upgrade = "django-upgrade --target-version 4.2 {args}"
run-all-django-upgrade = "git ls-files -- '*.py' | xargs django-upgrade --target-version 4.2"
python = [
  "run-pyright",
  "run-format",
  "run-reorder-python-imports",
  "run-autoflake",
  "run-ruff",
  "run-all-pyupgrade",
  "run-all-django-upgrade",
]
templates = ["run-curlylint", "run-djhtml"]
all = ["python", "templates"]

[tool.pyright]
typeCheckingMode = "basic"

[tool.curlylint]
include = '\.(html|jinja)$'
exclude = '''
(
  /(
      \.eggs           # exclude a few common directories in the root of the project
    | \.git
    | \.venv
    | \.ruff_cache
  )/
  | webpack-stats.html # also separately exclude a file named webpack-stats.html in the root of the project
)
'''

[tool.curlylint.rules]
indent = 4
aria_role = true
django_forms_rendering = true
image_alt = true
meta_viewport = true
no_autofocus = true
tabindex_no_positive = true

[tool.coverage.run]
source_pkgs = ["vagabond", "tests"]
branch = true
parallel = true
omit = ["src/vagabond/__about__.py"]

[tool.coverage.paths]
vagabond = ["src/vagabond", "*/vagabond/src/vagabond"]
tests = ["tests", "*/vagabond/tests"]

[tool.coverage.report]
exclude_lines = ["no cov", "if __name__ == .__main__.:", "if TYPE_CHECKING:"]

The project is just a test project for me, don't bother trying to make it work

juftin commented 8 months ago

Hmmm... try deleting your requirements.txt file and let hatch-pip-compile regenerate it. On my end, I wasn't able to run anything because the default lockfile (requirements.txt) already existed but was generated with pip-compile and not hatch-pip-compile - so it didn't include the expected header.

I also ran into issues with my machine on psycopg[c] so I just removed that in my test to get everything to work.

Tobi-De commented 8 months ago

Hmmm... try deleting your requirements.txt file and let hatch-pip-compile regenerate it. On my end, I wasn't able to run anything because the default lockfile (requirements.txt) already existed but was generated with pip-compile and not hatch-pip-compile - so it didn't include the expected header.

I also ran into issues with my machine on psycopg[c] so I just removed that in my test to get everything to work.

I had the issue before I tried with pip-compile directly to generate a requirements.txt file. Something weird just happened, the freshly generated hatch project that I was talking about just now, it was working fine, but I don't know how I managed to recreate the issue and I can't get it to work anymore

https://github.com/Tobi-De/fuko

juftin commented 8 months ago

What's the issue you're running into with fuko. I cloned the project, ran hatch run python --version and it created the lockfile no issue.

Tobi-De commented 8 months ago

What's the issue you're running into with fuko. I cloned the project, ran hatch run python --version and it created the lockfile no issue.

please try with this exact content in the pyproject file

[build-system]
requires = ["hatchling"]
build-backend = "hatchling.build"

[project]
name = "fuko"
dynamic = ["version"]
description = ''
readme = "README.md"
requires-python = ">=3.11"
license = "MIT"
keywords = []
authors = [
  { name = "Tobi DEGNON", email = "tobidegnon@proton.me" },
]
classifiers = [
  "Development Status :: 4 - Beta",
  "Programming Language :: Python",
  "Programming Language :: Python :: 3.8",
  "Programming Language :: Python :: 3.9",
  "Programming Language :: Python :: 3.10",
  "Programming Language :: Python :: 3.11",
  "Programming Language :: Python :: 3.12",
  "Programming Language :: Python :: Implementation :: CPython",
  "Programming Language :: Python :: Implementation :: PyPy",
]
dependencies = [
  "django",
]

[project.urls]
Documentation = "https://github.com/unknown/fuko#readme"
Issues = "https://github.com/unknown/fuko/issues"
Source = "https://github.com/unknown/fuko"

[tool.hatch.env]
requires = [
    "hatch-pip-compile"
]

[tool.hatch.envs.production]
type = "pip-compile"
dependencies = [
  "coverage[toml]>=6.5",
  "pytest",
]
lock-filename = "requirements.txt"

[tool.hatch.version]
path = "src/fuko/__about__.py"

[tool.hatch.envs.default]
type = "pip-compile"
dependencies = [
  "coverage[toml]>=6.5",
  "pytest",
]
lock-filename = "locks/default.lock"

[tool.hatch.envs.default.scripts]
test = "pytest {args:tests}"
test-cov = "coverage run -m pytest {args:tests}"
cov-report = [
  "- coverage combine",
  "coverage report",
]
cov = [
  "test-cov",
  "cov-report",
]

[[tool.hatch.envs.all.matrix]]
python = ["3.8", "3.9", "3.10", "3.11", "3.12"]

[tool.hatch.envs.types]
dependencies = [
  "mypy>=1.0.0",
]
[tool.hatch.envs.types.scripts]
check = "mypy --install-types --non-interactive {args:src/fuko tests}"

[tool.coverage.run]
source_pkgs = ["fuko", "tests"]
branch = true
parallel = true
omit = [
  "src/fuko/__about__.py",
]

[tool.coverage.paths]
fuko = ["src/fuko", "*/fuko/src/fuko"]
tests = ["tests", "*/fuko/tests"]

[tool.coverage.report]
exclude_lines = [
  "no cov",
  "if __name__ == .__main__.:",
  "if TYPE_CHECKING:",
]
Tobi-De commented 8 months ago

This is the error I get on my end, but I'm starting to think the issue is with my system for some reason

─────────────────────────────── Traceback (most recent call last) ────────────────────────────────╮
│ /Users/tobi/Library/Application                                                                  │
│ Support/pyapp/hatch/554652334309206614/1.8.0/lib/python3.11/site-packages/hatch/cli/__init__.py: │
│ 221 in main                                                                                      │
│                                                                                                  │
│   218                                                                                            │
│   219 def main():  # no cov                                                                      │
│   220 │   try:                                                                                   │
│ ❱ 221 │   │   return hatch(prog_name='hatch', windows_expand_args=False)                         │
│   222 │   except Exception:  # noqa: BLE001                                                      │
│   223 │   │   from rich.console import Console                                                   │
│   224                                                                                            │
│                                                                                                  │
│ /Users/tobi/Library/Application                                                                  │
│ Support/pyapp/hatch/554652334309206614/1.8.0/lib/python3.11/site-packages/click/core.py:1157 in  │
│ __call__                                                                                         │
│                                                                                                  │
│ /Users/tobi/Library/Application                                                                  │
│ Support/pyapp/hatch/554652334309206614/1.8.0/lib/python3.11/site-packages/click/core.py:1078 in  │
│ main                                                                                             │
│                                                                                                  │
│ /Users/tobi/Library/Application                                                                  │
│ Support/pyapp/hatch/554652334309206614/1.8.0/lib/python3.11/site-packages/click/core.py:1688 in  │
│ invoke                                                                                           │
│                                                                                                  │
│ /Users/tobi/Library/Application                                                                  │
│ Support/pyapp/hatch/554652334309206614/1.8.0/lib/python3.11/site-packages/click/core.py:1688 in  │
│ invoke                                                                                           │
│                                                                                                  │
│ /Users/tobi/Library/Application                                                                  │
│ Support/pyapp/hatch/554652334309206614/1.8.0/lib/python3.11/site-packages/click/core.py:1434 in  │
│ invoke                                                                                           │
│                                                                                                  │
│ /Users/tobi/Library/Application                                                                  │
│ Support/pyapp/hatch/554652334309206614/1.8.0/lib/python3.11/site-packages/click/core.py:783 in   │
│ invoke                                                                                           │
│                                                                                                  │
│ /Users/tobi/Library/Application                                                                  │
│ Support/pyapp/hatch/554652334309206614/1.8.0/lib/python3.11/site-packages/click/decorators.py:45 │
│ in new_func                                                                                      │
│                                                                                                  │
│ /Users/tobi/Library/Application                                                                  │
│ Support/pyapp/hatch/554652334309206614/1.8.0/lib/python3.11/site-packages/hatch/cli/env/prune.py │
│ :8 in prune                                                                                      │
│                                                                                                  │
│    5 @click.pass_obj                                                                             │
│    6 def prune(app):                                                                             │
│    7 │   """Remove all environments."""                                                          │
│ ❱  8 │   app.ensure_environment_plugin_dependencies()                                            │
│    9 │                                                                                           │
│   10 │   environment_types = app.plugins.environment.collect()                                   │
│   11                                                                                             │
│                                                                                                  │
│ /Users/tobi/Library/Application                                                                  │
│ Support/pyapp/hatch/554652334309206614/1.8.0/lib/python3.11/site-packages/hatch/cli/application. │
│ py:162 in ensure_environment_plugin_dependencies                                                 │
│                                                                                                  │
│   159 │   │   │   │   self.abort(code=first_error_code)                                          │
│   160 │                                                                                          │
│   161 │   def ensure_environment_plugin_dependencies(self) -> None:                              │
│ ❱ 162 │   │   self.ensure_plugin_dependencies(                                                   │
│   163 │   │   │   self.project.config.env_requires_complex, wait_message='Syncing environment    │
│   164 │   │   )                                                                                  │
│   165                                                                                            │
│                                                                                                  │
│ /Users/tobi/Library/Application                                                                  │
│ Support/pyapp/hatch/554652334309206614/1.8.0/lib/python3.11/site-packages/hatch/cli/application. │
│ py:177 in ensure_plugin_dependencies                                                             │
│                                                                                                  │
│   174 │   │   │   from hatch.utils.env import PythonInfo                                         │
│   175 │   │   │                                                                                  │
│   176 │   │   │   management_command = os.environ['PYAPP_COMMAND_NAME']                          │
│ ❱ 177 │   │   │   executable = self.platform.check_command_output([app_path, management_comman   │
│   178 │   │   │   python_info = PythonInfo(self.platform, executable=executable)                 │
│   179 │   │   │   if dependencies_in_sync(dependencies, sys_path=python_info.sys_path):          │
│   180 │   │   │   │   return                                                                     │
│                                                                                                  │
│ /Users/tobi/Library/Application                                                                  │
│ Support/pyapp/hatch/554652334309206614/1.8.0/lib/python3.11/site-packages/hatch/utils/platform.p │
│ y:122 in check_command_output                                                                    │
│                                                                                                  │
│   119 │   │   kwargs.setdefault('stderr', self.modules.subprocess.STDOUT)                        │
│   120 │   │   self.populate_default_popen_kwargs(kwargs, shell=shell)                            │
│   121 │   │                                                                                      │
│ ❱ 122 │   │   process = self.modules.subprocess.run(self.format_for_subprocess(command, shell=   │
│   123 │   │   if process.returncode:                                                             │
│   124 │   │   │   self.__display_func(process.stdout.decode('utf-8'))                            │
│   125 │   │   │   self.exit_with_code(process.returncode)                                        │
│                                                                                                  │
│ /Users/tobi/Library/Caches/pyapp/distributions/_554652334309206614/python/lib/python3.11/subproc │
│ ess.py:548 in run                                                                                │
│                                                                                                  │
│    545 │   │   kwargs['stdout'] = PIPE                                                           │
│    546 │   │   kwargs['stderr'] = PIPE                                                           │
│    547 │                                                                                         │
│ ❱  548 │   with Popen(*popenargs, **kwargs) as process:                                          │
│    549 │   │   try:                                                                              │
│    550 │   │   │   stdout, stderr = process.communicate(input, timeout=timeout)                  │
│    551 │   │   except TimeoutExpired as exc:                                                     │
│                                                                                                  │
│ /Users/tobi/Library/Caches/pyapp/distributions/_554652334309206614/python/lib/python3.11/subproc │
│ ess.py:1026 in __init__                                                                          │
│                                                                                                  │
│   1023 │   │   │   │   │   self.stderr = io.TextIOWrapper(self.stderr,                           │
│   1024 │   │   │   │   │   │   │   encoding=encoding, errors=errors)                             │
│   1025 │   │   │                                                                                 │
│ ❱ 1026 │   │   │   self._execute_child(args, executable, preexec_fn, close_fds,                  │
│   1027 │   │   │   │   │   │   │   │   pass_fds, cwd, env,                                       │
│   1028 │   │   │   │   │   │   │   │   startupinfo, creationflags, shell,                        │
│   1029 │   │   │   │   │   │   │   │   p2cread, p2cwrite,                                        │
│                                                                                                  │
│ /Users/tobi/Library/Caches/pyapp/distributions/_554652334309206614/python/lib/python3.11/subproc │
│ ess.py:1950 in _execute_child                                                                    │
│                                                                                                  │
│   1947 │   │   │   │   │   │   err_filename = orig_executable                                    │
│   1948 │   │   │   │   │   if errno_num != 0:                                                    │
│   1949 │   │   │   │   │   │   err_msg = os.strerror(errno_num)                                  │
│ ❱ 1950 │   │   │   │   │   raise child_exception_type(errno_num, err_msg, err_filename)          │
│   1951 │   │   │   │   raise child_exception_type(err_msg)                                       │
│   1952                                                                                           │
│   1953                                                                                           │
╰──────────────────────────────────────────────────────────────────────────────────────────────────╯
FileNotFoundError: [Errno 2] No such file or directory: '1'
ofek commented 8 months ago

You need a binary from this release or onward https://github.com/pypa/hatch/releases/tag/hatch-v1.8.1

Tobi-De commented 8 months ago

You need a binary from this release or onward pypa/hatch@hatch-v1.8.1 (release)

Thank you, @ofek. It now works. However, considering that the hatch installation method could impact the overall user experience, I believe it would be beneficial to include a note in the readme. What are your thoughts, @juftin?

ofek commented 8 months ago

The first binaries were introduced in 1.8.0 and the automatic installation of environment plugins was fixed in the next patch release so I'm not sure it would be worth it to document.

Up to Justin of course

Tobi-De commented 8 months ago

The first binaries were introduced in 1.8.0 and the automatic installation of environment plugins was fixed in the next patch release so I'm not sure it would be worth it to document.

Up to Justin of course

I had this issue with the latest version 1.9.1, and I think the first time I tried to install it, my first option on macos was the install script (I upgrade with hatch self update), I think it's worth pointing out, for the 3 people (including me) who might get lost, and give up.

In any case, the ux hatch + hatch-pip-compile is simply magnificent, thanks @ofek and @juftin.

ofek commented 8 months ago

Yeah the self update wouldn't work because that just does the Python part so that's probably where the confusion is. I thought about somehow figuring out a cross-platform way to upgrade the actual binary but it's very difficult on Windows.