Using editable installs with `hatch`

[NellyWhads]

Describe the bug When I create uv hatch-backed project and install it, all modules within the regular package structure (ie. within the project directory) install as editable. However, all bricks do not install in editable mode. This requires me to constantly run uv sync --reinstall-module {project-name} along with any other dep groups to update the sources in the .venv and then re-start my python notebooks to pick up the change. This is rather annoying since the %autoreload decorators don't work properly, and I have to basically re-build the wheel on every change to a component. This was not the behavior when using the poetry plugin.

To Reproduce Steps to reproduce the behavior:

1. Use a polylithic config as exemplified in this repo backed by uv and hatch
2. Install the project using uv's default methods (ie. uv sync {args})
3. Make a change to a brick (I currently only use components) like adding a new class
4. Attempt to access the new class from a script or an interactive uv run python/uv run ipython session 4.1. Activate the uv .venv manually and try again 4.2. Use a code editor like VSCode to trace the location of the module and find it to be in .venv/lib/{python-version}/site-package/{namespace}/{module} instead of the expected {workspace}/components/{namespace}/{module} 4.3. Be sad
5. Update the sources using uv sync --reinstall-module {project-name}
6. Try step 4. again, this time, skipping 4.3.

Error Log No logs available.

Expected behavior I expect editable access to the bricks along with the project.

Desktop (please complete the following information):

• OS: MacOS Sonoma Ubuntu 20.04 Ubuntu 22.04 Ubuntu 24.04
• Python version: 3.8-3.11
• UV version: 0.5

[NellyWhads]

Moving this to the appropriate repo - apologies. https://github.com/DavidVujic/poetry-multiproject-plugin/issues/68

[DavidVujic]

I think it's the appropriate repo! The multiproject is for Poetry users.

[DavidVujic]

@NellyWhads Thank you for reporting!

This should work as you expect, and I quickly tried with adding a class to an existing component in the uv polylith example repo.

Without doing anything more than add the class, I started a REPL using uv run ipython and imported the class without any issues. How does the root pyproject.toml look like in your repo? I think it maybe misses something in there?

[DavidVujic]

Oh wait, do you have Python modules in the Project directory?

[NellyWhads]

Yes i do, I build a python module in the project for the "development" cycle of a new project. If the project maintainer decides to migrate functionality to a "common" brick, they can choose to do so. But I believe the polylith tooling should not break standard python packaging workflows (ie. source code in multiple include-able directories in the project directory).

[DavidVujic]

Can you share an example of the structure of the components, bricks and projects (and what's in the projects)?

I don't understand what you mean by building a python module, can you share some more of your workflow?

[NellyWhads]

Of course. Basically, I would like the project to have the ability to house its own source code, in addition to the source code provided by the bricks.

Example

Namespace: my_namespace

Components:

• my_namespace.datasets
• my_namespace.tools
• my_namespace.models

Project:

• Name: my_project
• Uses my_namespace.datasets and my_namespace.models
• Adds code into the {git_root}/projects/my_project/my_project/* code path

When I do this, I get editable installations of the modules in my_project but not in my_namespace.

I understand that there is a namespace inconsistency between components and the project, however, this shouldn't break editable installs, should it?

[DavidVujic]

Aren't editable installations the local libraries some use in their repos, such as code in folders with their own pyproject.toml and deps?

The components and bases should just be included code. Do you run a uv command to install the project? In a Polylith Workspace, there is one .venv at the root, and the repo is aware of the components and bases without any installs (in the same way as a single project and a src folder).

[DavidVujic]

Oh, maybe you run the uv sync --reinstall-packages command. I guess that will create a local project-specific .venv or somehow include the entire project in the root environment. Which part in this is about Polylith and the Polylith tool? It doesn't sound like the tool is used here, right?

[NellyWhads]

yes, this is the issue - i need to keep running uv sync --reinstall-packages to make this work.

I believe it's an issue with polylith because the package sources (ie. code from the my_project module) loads just fine as editable, however, code from the my_namespace does not.

The tool is used to automatically include the bricks (i guess at build time?). I'm wondering it the polylith tool can be extended to patch this workflow so that I don't have to keep running the reinstall command to pick up every edit.

I keep the .venv in the project directory, since I may be working on multiple projects at a time. The .venv location, I don't think should make a difference, but perhaps it does.

[DavidVujic]

Thank you for explaining the setup!

I think this is caused by the install of the project into the virtual environment. That would explain why the bricks end up in a diferent place than expected (the project + its code is installed as a library and not referenced as the root-level components).

This setup is not the expected workflow for Polylith, and I would recommend that you rethink the parts where code is put in the projects and where the projects are installed in a separate virtual environment.

The development workflow is meant to be happening at the workspace root, where the top-level .venv contain the third-party deps and where to find the paths to the local source code. When you navigate around the code and go-to-definition for a brick, it shouldn't find them in the .venv but in the components or bases folders.

The root pyproject.toml should have all the third-party deps and the bricks added, and it's from that environment you can access all code via the REPL or a Jupyter notebook.

The projects folders are meant to be used when deploying, that's when you run the uv build or poetry build-project commands that will create wheels that can be installed in a production environment. That's the main purpose of the project-specific pyproject.toml files.

With this setup, the polylith tool will recognize all the bricks, and where they are used. It will also notify on any missing dependencies or bricks in a specific project.

[DavidVujic]

If you need any help or suggestions on how to set up your repo as I recommend in the comment above, I can help you with that. I like that you use uv and it fits really well with Polylith.

[NellyWhads]

Okay, let's perhaps chat more interactively on Monday?

I require that this works with the full feature set of uv (including the fact that the location of the .venv should not impact the behavior of the tooling)

[NellyWhads]

As for the dev setup - I may just be in a particular edge case situation - I'm using the uv setup for one project, but have to wait a few weeks until I transition the whole repo to uv.

In either way, if the tooling does not "need" to break for this set up, it would be super helpful.

[DavidVujic]

Sounds like a good idea to chat on monday!

If you haven't seen this already, here's an example uv and Polylith setup: https://github.com/DavidVujic/python-polylith-example-uv

[DavidVujic]

If you want to try things out until monday, I think the problem happens with the project-specific pyproject.toml. I don't think the Polylith-recommended project-setup will work in the way you use it.

That setup is meant to be used when building wheels, for local development you would probably need something more like the way the root pyproject.toml is configured. But again, it is not the recommended way and will likely cause issues when building wheels for deployment.

[NellyWhads]

Sounds like a good idea to chat on monday!

If you haven't seen this already, here's an example uv and Polylith setup: https://github.com/DavidVujic/python-polylith-example-uv

This is what I based the project off of. I know I'm not using the repo-wide config, but hopefully that changes soon.

[DavidVujic]

I've experimented a bit with this and can verify that the project-specific pyproject.toml with the Hatch/uv kind of setup doesn't support your workflow.

To run a .venv for the project only, and to have the local source code folder play nice with the Polylith bricks, you would need to change the pyproject.toml in the specific project into something like this:

# Disable the build-specific features of Hatch/uv, that is also used by the "uv sync" command
# [tool.polylith.bricks]
# "../../components/my_namespace/my_brick" = "my_namespace/my_brick"

# add knowledge in the environment about where to find the source code
[tool.hatch.build]
dev-mode-dirs = ["../../components", "../../bases"]

If you would run uv sync in the project folder, you will get a local .venv and you won't need to re-install or re-sync anything. Changing a brick or changing a local project-specific source code will be picked up by IPython without any need of restarting or reinstalling.

This setup will work for your specific development flow, but in a CI, or when building a wheel, you will have to restore the pyproject.toml as before.

I would still recommend that you have all Python code in the bases and components folders. You will have access to the bricks from the root .venv without any extra installs or sync. And not developing in the individual projects with a separate .venv.

I think the suggestions here will solve your issue and why it happens. For Poetry, it was a bit different (mainly because the uv sync command does more things behind the scenes together with the Hatchling backend, than the poetry install command).