Open bgotink opened 4 years ago
I think what you're looking for would be a kind of utility library built on top of the Yarn libraries. While I think it would be quite valuable, I'm not certain it would be a good fit for the core project, as I don't see direct use case for us as consumers. I'd prefer to see that live in a separate repository and see how it evolves over time.
This could definitely be built in a separate repository. There are only two changes—as I see it atm—that really must happen in the yarn repository:
@yarnpkg/cli/package.json
available to plugins (and pin the version numbers at build time so the manifest doesn't list a dependency on @yarnpkg/core
with range workspace:^2.1.1
)BaseCommand
and WorkspaceRequiredError
from @yarnpkg/cli
to another package. These two classes are the reason a lot of the plugins have peerDependencies on the CLI package, and removing that would make installing @yarnpkg/cli
to consume a plugin's node API no longer necessary.make
@yarnpkg/cli/package.json
available to plugins (and pin the version numbers at build time so the manifest doesn't list a dependency on@yarnpkg/core
with rangeworkspace:^2.1.1
)
I'm not sure I understand why it would be a problem, especially since the core is a peer dependency of the CLI, so you always have control on the version you use (and if you use the wrong one, you'll get a peer dependency warning).
move
BaseCommand
andWorkspaceRequiredError
from@yarnpkg/cli
to another package. These two classes are the reason a lot of the plugins have peerDependencies on the CLI package, and removing that would make installing@yarnpkg/cli
to consume a plugin's node API no longer necessary.
Is it really a problem though? 🤔 The rest of the CLI is fairly small, I wonder if that's really useful to split the builtin CLI configuration from the CLI itself.
I'm not sure I understand why it would be a problem, especially since the core is a peer dependency of the CLI, so you always have control on the version you use (and if you use the wrong one, you'll get a peer dependency warning).
If I'm using the packages of yarn 2.5 but the user has yarn 2.1 installed on their system, making changes to the dependencies could have adverse effects. Examples:
.pnp.js
file will probably look ever so slightly different because of changes to support package exports, or due to changes in fslibAt the very least these cause unwanted changes on disk.
At worst these break builds (e.g. if yarn install --check-cache
is part of the project's CI/CD pipeline)
Is it really a problem though? 🤔 The rest of the CLI is fairly small, I wonder if that's really useful to split the builtin CLI configuration from the CLI itself.
The CLI does contain dependencies on all plugins, which can easily lead to trouble. For example:
I'm using @yarn/cli
version 2.1.x, while in @yarn/cli
version 2.2 we introduce support for fallback npm registries (there's a ticket open for it somewhere). I update the @yarnpkg/plugin-npm
package to 2.2.
Everything looks okay at install time. Everything works fine at build time. But, if I use the npmHttpUtils
I get an error saying that npmFallbackRegistries
is an unknown configuration key.
Reason: the @yarn/cli
contains its own version of the @yarnpkg/plugin-npm
that's still at 2.1.x, and that version is used to parse the user's configuration.
TL;DR: Consuming yarn API's is hard and potentially dangerous. This ticket proposes solutions to both problems.
Describe the user story
At the moment it's hard to import yarn packages and use the API. Not only because of a lack of useful documentation (#1654) but also the API design itself.
Here's an example program to fetch information from the user's configured NPM registry regarding a package, with inline comments explaining why the API is unergonomic
On top of the comments describing issues above, there's also a lack of validation that my imported version of the yarn packages is compatible with the ones installed on the user's system. In the code above I doubt this would be a problem per se, as I'm only using the yarn API to query things. But, it would be entirely possible to use the yarn API to modify manifests or change resolutions or change the configuration, and then commit these changes to disk, potentially stopping the yarn command from working.
Describe the solution you'd like
I can actually think of multiple solutions here, all of which have upsides and downsides. In the end I would like to have:
Option 1: make the yarn binary requirable
This actually makes validating the version compatibility: is the version of the package in the yarn bundle semver compatible with
^<version of the API package I've installed in my library>
. Using the API would also guarantee you're not left with a broken system, as long as you're doing valid things with the API. We would for instance never end up with a yarn lockfile with a version number that the yarn bundle in the repo doesn't understand.It would complicate the yarn bundle quite a bit. We would need to ensure all imports are available via their proper name. In the end this will definitely have a negative impact on the bundle size.
Option 2: create a more ergonomic API package
Side note: I'm going to use the package name
@yarnpkg/flurb
here, because I'm bad at naming things and I don't want the name to distract from the content.We could create a package called
@yarnpkg/flub
that does a couple of things to facilitate fetching the configuration:Configuration.find
Configuration.find
with a defaultstrict: false
and the plugin configuration mentioned in the previous bullet.Configuration.find
to__dirname
So the code sample above would be reduced to
which already looks a lot better.
This leaves the issue of the possible incompatibilities though. We could solve these in a number of ways, but the only one I can think of that catches all possible incompatibilities is the following one: we create a new resolver. Let's call that one "flurb" as well, because I am not in an imaginative mood. My package would have the following manifest:
The yarn binary would resolve these versions as follows:
npm:<the exact version number>
The biggest issue with this approach would be that it's not possible to install my package using any other package manager. Maybe this could be solved using a field in dependenciesMeta instead of a new range protocol? Or maybe we could magically resolve these builtin packages to the builtin version number in the npm resolvers?
An alternative to this entire approach would be to make the yarn bundle return a map of package names to version numbers when required, instead of executing. We could then validate compatibility between the version in my library and the version in the bundle, but unless the version is an exact match there's always a risk of incompatibilities.