Order of lock file constraints

[FlorianWilhelm]

@juftin Thanks for the #12 feature and awesome logo btw.

I am just playing around a bit with it and I noticed that I can create a lock file for a test environment, which depends on the default environment, without the lock file for the default environment being created. Does that make sense?

I would assume that when running e.g. hatch run test:cov and the when the test environment is created, it should realise that there is yet no default environment from which it inherits so it should first create a lock file for default.

So if I am able to create lock file for test and maybe ours later create the environment for default, it could mean that default is not consistent with test anymore, right?

[juftin]

Hey @FlorianWilhelm, I think you are maybe working with an outdated version of hatch-pip-compile? Hopefully you're just experiencing some pain of being an early-adopter 🙃

Here is an example config (with the most recent version pinned):

[build-system]
build-backend = "hatchling.build"
requires = ["hatchling"]

[project]
version = "0.0.0"
name = "hatch-pip-compile-test"

[tool.hatch.env]
requires = ["hatch-pip-compile>=1.2.0"]

[tool.hatch.envs.default]
type = "pip-compile"
dependencies = ["packaging"]
pip-compile-constraint = "default"

[tool.hatch.envs.test]
dependencies = [
  "pytest"
]

[tool.hatch.envs.test.scripts]
test = "pytest --help"

Next I runhatch run test:test and it raises the following error:

LockFileNotFoundError: [hatch-pip-compile] The lock file /Users/justinflannery/test/hatch-test-1/requirements.txt does not exist. Please create it: `hatch env create default`

[juftin]

With the latest version of the tool it will both check that the upstream lock file exists and check that it is up to date too. I decided not to implement the ability to generate the missing lock file automatically because it's possible that environment A and environment B could be running on different interpreters / Python versions - so only environment B should be able to generate its own lockfile.

---

There is still one situation this doesn't cover though:

1. You create the default lockfile
2. You create the test lockfile with the default lockfile as its constraint
3. You update the default lockfile without actually changing the requirements specified in your pyproject.toml
4. You run the test environment and it thinks it's up to date but remains pinned on the old versions

I'll file an issue for this - we can fix this issue by storing/verifying a checksum of the default lockfile inside the header of the test lockfile

[FlorianWilhelm]

Thanks @juftin, somehow I had messed the configuration up but now I am seeing the same error as you when I try to create the test environment before the default environment is created.

Do you think instead of showing this error message that one could create the default environment (or recursively the environments, the called environments inherits from) for the sake of usability?

I am trying to incorporate this into my hatch cookiecutter template, i.e. the hatchlor, to allow everyone to make easier use of your locking feature.

[juftin]

Do you think instead of showing this error message that one could create the default environment (or recursively the environments, the called environments inherits from) for the sake of usability?

This is a tricky one. The plugin currently runs inside the Python interpreter of a given environment. I'm able to figure out whether the upstream lockfile exists or not but the problem I run into is being able to use the actual interpreter of that upstream environment to actually invoke pip-compile. There is probably a way to do this - I just have to figure out how to do that. (subprocess.run(["hatch", "env", "run", "--env", "upstream-env", "--", "python", "--version"]) would do this, but I'm hoping to have a nicer solution that invoking hatch from hatch)

[juftin]

I am trying to incorporate this into my hatch cookiecutter template, i.e. the hatchlor, to allow everyone to make easier use of your locking feature.

I like how you did this on the hatchlor:

[tool.hatch.envs.default.scripts]
upgrade-all = "PIP_COMPILE_UPGRADE=1 hatch env run --env {env_name} -- python --version"
upgrade-pkg = "PIP_COMPILE_UPGRADE='{args}' hatch env run --env {env_name} -- python --version"

So something like hatch run docs:upgrade-all works if it inherits the default template?

[juftin]

Closing this as complete. I created #25 to track the auto-creation of upstream lockfiles - that has an incoming fix too!

[FlorianWilhelm]

Yes, it seems hatch run docs:upgrade-all to work. I'll let you know when I now more. So far I released the hachtlor 0.3 with hatch-pip-compile included. Let's see what the feedback is :-)