Several crates per repository (monorepos)

[mosteo]

I'm creating this issue to track this idea which keeps popping up so interested parties can track it / chime in with comments.

For the sake of clarity: a monorepo is a repo that contains several crates, usually related to each other. I'll call a subcrate each crate within a monorepo.

The issue is that, for convenience or other reasons, some people keep several possibly interrelated crates in a single repository. Publishing each one is currently a tedious affair as this requires zipping each crate independently.

Possibilities that have been discussed/attempted in the past:

1. Split your repositories. People have manifested this is not really desirable (example). For new projects is still a valid approach.
2. Specialized parent/child crates. This introduces unwanted complexity, and I don't think this is the way anymore. (#108 #200 #349).
3. Add an extra attribute to repo origins so the crate can be located anywhere within a repo, not just at the root.
   • This would allow to keep our publish strategy.
   • Each crate in a monorepo would be just a plain regular crate with its manifest.
   • As long as crates were depended upon from the same commit (e.g., same version), there isn't even wasting of space.
4. Simplify tar+upload during publishing, so no other changes are required in Alire.
5. Introduce a meta-manifest that points to the crates in the repo, to streamline publishing of several crates at the same time.

This is orthogonal to pins (#721), that can be used freely in local development to use all/part of the crates in the monorepo simultaneously. Pins are not used for publishing.

My personal standing right now is that

• 3 could (hopefully) be a minor change that would allow simple publishing of crates, even if you don't want to publish all of them simultaneously (think of crates that are not versioned in lockstep), hence simple and flexible.
• 4 is too much work that is not really needed if 3) works well without unexpected interactions.
  • OTOH there are people that prefer tarballs to repositories.
• 5 is doable after 3 to automate publishing of crates in batch.

Other issues that touched on related aspects: #350

[onox]

@mosteo I've a small problem with the pin circularity detection (using af2bead of alire): in the orka crate I get this error:

error: Pin circularity detected when adding pin orka_transforms --> orka_simd:
error:    Last manifest in the cycle is /path/to/orka/orka_transforms/alire.toml

orka_transforms depends on orka_simd (and therefore orka_transforms/alire.toml has a pin on orka_simd), and orka itself directly depends on both orka_transforms and orka_simd. In orka/alire.toml file I have a pin to orka_simd and orka_transform. So there are no actual cycles if the edges are directional (DAG):

orka -> orka_transforms -> orka_simd
   \___________________________^

I can fix the error above by removing the pin on orka_simd from orka/alire.toml, but then the compilation of orka succeeds only as long as orka_transforms still depends on orka_simd.

[mosteo]

@onox, thanks for the report, that is certainly not intended.