Should the registry admit duplicate locations?

thomashoneyman commented 2 years ago

I noticed while working on #444 that a number of packages in the registry point to the same repository.

For example, we uploaded several releases for the profunctors library by @joneshf, but it turns out that this package is just a fork of the profunctor library using another name. It doesn't have any releases of its own and is listed in the Bower registry as pointing at the purescript-profunctor core library. The result: the profunctors library in the registry is just a bunch of copies of the profunctor library.

Worse, these duplicate libraries we're registering (such as profunctors) aren't even installable by Bower, so I'm not sure why they're included in the Bower registry file:

λ bower install purescript-profunctors       
bower ENOTFOUND     Package purescript-profunctors not found

Should we allow multiple packages in the registry to point to the same location? For example, should I be able to register a new package, affy, which points to https://github.com/purescript-contrib/purescript-aff? Right now we allow this and will package up every aff release as an affy release.

My feeling is that we don't want new packages to be able to target an already-published package location. We definitely don't want a new package to be registered against a package location that is currently in use; possibly, if a location was used and then the package was transferred, it could become eligible for use again, but I think we should deny that as well.

Here is the list of duplicates in the new-packages.json and bower-packages.json files. This list represents package names that are all registered to the same package location.

{
  "https://github.com/arthurxavierx/purescript-carpenter.git": [
    "carpenter",
    "react-carpenter",
    "react-cedar"
  ],
  "https://github.com/chrisdotcode/purescript-requests.git": [
    "http-request",
    "requests",
    "responseable"
  ],
  "https://github.com/fehrenbach/purescript-unordered-collections.git": [
    "unordered-collections",
    "unordered-containers"
  ],
  "https://github.com/jasonzoladz/purescript-rxjs.git": [
    "rx-observable",
    "rxjs"
  ],
  "https://github.com/joneshf/purescript-path.git": [
    "path",
    "unix"
  ],
  "https://github.com/lumihq/purescript-react-dnd-basic.git": [
    "react-basic-dnd",
    "react-dnd-basic"
  ],
  "https://github.com/mitchdzugan/purescript-impulse.git": [
    "impulse",
    "my-project"
  ],
  "https://github.com/natefaubion/purescript-heterogeneous.git": [
    "heterogeneous",
    "heterogenous"
  ],
  "https://github.com/nsaunders/purescript-dotenv.git": [
    "dotenv",
    "node-dotenv"
  ],
  "https://github.com/nullobject/purescript-digraph.git": [
    "dgraph",
    "digraph"
  ],
  "https://github.com/opyapeus/purescript-emo8.git": [
    "emo8",
    "nemo"
  ],
  "https://github.com/paf31/purescript-hoist.git": [
    "functor-optics",
    "hoist"
  ],
  "https://github.com/paluh/purescript-pux-clappr.git": [
    "clappr-pux",
    "pux-clappr"
  ],
  "https://github.com/purescript-hyper/purescript-hypertrout.git": [
    "hyper-routing-server",
    "hypertrout"
  ],
  "https://github.com/purescript-hyper/purescript-trout-client.git": [
    "hyper-routing-xhr",
    "trout-client"
  ],
  "https://github.com/purescript-hyper/purescript-trout.git": [
    "hyper-routing",
    "trout"
  ],
  "https://github.com/purescript/purescript-profunctor.git": [
    "profunctor",
    "profunctors"
  ],
  "https://github.com/sharkdp/purescript-sparkle.git": [
    "flarecheck",
    "sparkle"
  ],
  "https://github.com/slamdata/purescript-sql-squared.git": [
    "sql-squared",
    "sqlsquare"
  ],
  "https://github.com/wfaler/purescript-jquery-ajax.git": [
    "client-ajax",
    "jquery-ajax"
  ]
}

Next Steps

I think we should address this situation by taking these steps:

Adjust the new-packages.json and bower-packages.json files so that there aren't duplicate package locations anymore. The packages I've spot-checked all have a clear canonical package name <-> package location.
Adjust the API pipeline and the verify-registry script check so that we don't allow packages to use package locations that are already being used. Right now we only store the current package location for any given package, so there's no way for us to enforce that the package location you're trying to register was never used before. We'd have to adjust metadata so that it stores a package location per version if we wanted to enforce that.

f-f commented 2 years ago

This seems the right path forward, I have a few notes:

we have been allowing duplicate locations because of the monorepo support - i.e. multiple packages could share a location URL as they could be stored in the same repo. We can have locations be unique as long as we include the subdir in this equation - i.e. the tuple url, subdir must be unique. (so far we haven't been using the subdir at all, which is why we encountered this)
what's the reason for forbidding the use of old locations once they become free again? (i.e. no other package in the registry uses them). They are not like package names, in the sense that things are indexed by them, they are just places where to fetch stuff - if a location was used before but is not anymore, then there should be no danger of fetching the same content (because nothing is pointing at it anymore) and not enforcing this would let us skip a check and store additional data.

thomashoneyman commented 2 years ago

Ah! Of course; monorepo support requires duplicate locations, with different subdirectories. That’s a great point. So we would want to enforce uniqueness for the location + subdir.

As far as reusing old locations — you’re right, that’s not an issue. If you’re installing by package name then it doesn’t matter what the location was.

purescript / registry-dev

Should the registry admit duplicate locations? #445