astral-sh / rye

a Hassle-Free Python Experience
https://rye.astral.sh
MIT License
13.6k stars 466 forks source link

Monorepo approach leads to conflicting URLs for packages #1292

Open ghost opened 2 months ago

ghost commented 2 months ago

Hi, maybe my approach is bad but basically I want to manage a monorepo with rye.

I have several libraries, some of which depend on each other, but I want to keep the libraries separate (in my case they are git submodules). So it should be possible to install a library, let's called it package_1 from a remote location (e.g. GitHub) and if that library depends on another library that I manage, it should also fetch that library from the remote location (e.g. I have added a dependency with rye add package_2 --git=https://github.com/username/package_2).

In the case above my monorepo would look like this:

monorepo
  - libs
    - package_1
      - ...
      - pyproject.toml
    - package_2
      - ...
      - pyproject.toml
  - ...
  - pyproject.toml

Now as there are two sources in monorepo for this particular library package_2, locally at monorepo/libs/package_2 and remotely at https://github.com/username/package_2, I run into problems.

It seems rye cannot handle this use case, not sure if this is a bug (or missing feature) on the rye side or if my approach is bad. Any help would be appreciated!

Steps to Reproduce

  1. Create a monorepo rye init --virtual monorepo
  2. Create libs folder cd monorepo && mkdir libs
  3. Add libs rye init libs/package_1 && rye init libs/package_2
  4. Push one package_2 to a remote (e.g. GitHub)
  5. Add dependency to package_1: cd libs/package_1 && rye add package_2 --git=https://github.com/username/package_2
  6. Run rye sync

Expected Result

No errors, rye will use the local repository (or the remote) as default.

Actual Result

Reusing already existing virtualenv
Generating production lockfile: /home/username/monorepo/requirements.lock
error: Requirements contain conflicting URLs for package `package_2`:
- file:libs/package_2
- git+https://github.com/username/package_2
error: could not write production lockfile for workspace

Version Info

rye 0.37.0
commit: 0.37.0 (09b67c469 2024-07-20)
platform: linux (x86_64)
self-python: cpython@3.12.2
symlink support: true
uv enabled: true

Stacktrace

No response

Tremeschin commented 1 month ago

I vaguely remember trying something like this on my monorepo, try adding the main lib's git dependency as a --dev dependency on the submodules, and have the submodules on the [tool.rye.workspace] of the main pyproject.toml

I've then since moved to just warning that a repo shouldn't be cloned alone, and grab the latest PyPI version of every dependent package, at a risk of not working, but git dev deps should work if using rye as a manager (users would pip install . or poetry install submodules clone and won't work anyways due lack of consensus of how to define dev dependencies on pyproject.toml...)