Design a plugin frontend system to serve language level registry requests

[blaggacao]

Problem Statement

A nix-backed SDLC currently can't avoid the (let's call it) "media break" to at some point separately "publish" binary artifacts to some form of what is commonly known as registry: container registry, package registries, etc.

Some people already take offense with "media breaks" themselves (like me): they are a bad thing from first principles.

But a more vested argument is indeed that once we break the medium, we loose a lot of features by which we could otherwise make our registry more useful.

So this problem statement is about a missed opportunity. Not about an itch (or maybe missed opportunities itch you, too? Especially when Nix could raise and shine.).

Some of the things how your average registry could be augmented:

• seamless cross-language registry (same domain)
  • potentially implementable by nixpkgs to become the most comprehensive package registry in the world.
  • this may have a significant collateral impact on dx and developer acceptance of a nix based SDLC
• reproducability and software supply chain guarantees delivered by the very registry implementation (hello!?! Unheard of!)
• if sources can be durably cached, also: a software vault for all your releases (+ all deps)

Potential Solution

Design attic with a plugin system in mind that makes it possible (and maybe even easy?!) to amend it's state and frontend with language-specific registry functionality and routes.

Listings and nicely rendered package pages could make sense to become a practical alternative to existing registries for end users.

And what about NAR?

I'll quote @Ericson2314:

Don't worry at all, it's an implementation detail.

But in the short term some streaming transcoder or similar thing + some extra metadata in state might just get us going.

[blaggacao]

Re NAR

I'd also like to link https://github.com/flokli/nix-casync/issues/41 which questions NAR in suggesting to unpack it internally.

---

_{Sidenote (maybe for @Ericson2314 — who likely knows all this already): based on Lennart's introductory post about casync the catar format (with its focus on reproducability) combined with --without= flags stripping undesired file metadata could uncompromisingly reproduce the goals of NAR while at the same time bringing the benefits of casync (-based distribution) to the Nix ecosystem.}

_{It reads:}

As an extra twist, we introduce a well-defined, reproducible, random-access serialization format for file trees (think: a more modern tar), to permit efficient, stable storage of complete file trees in the system,

[yangm97]

I can see somebody else gets triggered by npm fetcher storing tarballs, extracting stuff into a fake cache and tem copying everything (again!) into one giant closure which takes ages to (re)build.

However, this is a nixpkgs issue IMHO.