search

oven-sh / bun

Incredibly fast JavaScript runtime, bundler, test runner, and package manager – all in one
https://bun.sh
Other
74.45k stars 2.78k forks source link

support patching dependencies, similar to `pnpm patch`/`patch-package` #2336

Closed josefaidt closed 5 months ago

josefaidt commented 1 year ago

What is the problem this feature would solve?

I am currently working with a few dependencies that will cause errors at runtime due to their imports, and I'd also like to apply general changes without suggesting changes to the projects.

Today I'm using pnpm to manage dependencies and bun to run the project, which is fine for the most part. Unfortunately patch-package will not work as it is looking for npm & yarn lockfiles.

What is the feature you are proposing to solve the problem?

bun patch

What alternatives have you considered?

No response

harrytran998 commented 1 year ago

Hello @josefaidt - Can you have some time to make this issue progress? Thank 😆

birkskyum commented 1 year ago

Alternatively, if it's easier to add support for using patch-package, that would go a long way too.

OnurGvnc commented 1 year ago

Attempting to create a patch using bunx patch-package <package-name> fails. The patch-package outputs the following error:

Copy code
**ERROR** No package-lock.json, npm-shrinkwrap.json, or yarn.lock file.

You must use either npm@>=5, yarn, or npm-shrinkwrap to manage this project's
dependencies.

However, if you have pre-existing patches in the ./patches directory, you can successfully apply them using the bunx patch-package command (or via package.json -> scripts -> "postinstall": "patch-package")

As a temporary workaround to generate a patches/package+name+version.patch file

I found a new workaround:

SimonVerhoeven commented 1 year ago

It would be nice to see this to facilitate https://github.com/angular/angular/issues/46719

glekner commented 1 year ago

Any plans to implement this? @Jarred-Sumner

deathemperor commented 1 year ago

I found a new workaround:

  • run bun install --yarn it just converts bun.lockb into a yarn.lock
  • make the necessary modifications inside node-modules/<package-name>
  • bunx patch-package <package-name>
  • package.json -> scripts -> "postinstall": "patch-package" ds300/patch-package

this works with one minor tweak: "postinstall": "bunx patch-package" if patch-package is not in your dependencies

juliusmarminge commented 1 year ago

+1 pnpm patch is no nice to have and definetely a blocking feature before adopting bun in some of my repos

juliusmarminge commented 1 year ago

I found a new workaround:

  • run bun install --yarn it just converts bun.lockb into a yarn.lock
  • make the necessary modifications inside node-modules/<package-name>
  • bunx patch-package <package-name>
  • package.json -> scripts -> "postinstall": "patch-package" ds300/patch-package

This wont work though for platforms like vercel that determines the package manager by which lockfile you have in the repo, so if you have a yarn.lock they will use yarn to install deps unless i put in manual overrides :/

intrnl commented 1 year ago

I'm very certain you only need the yarn.lock so that patch-package can determine what package to retrieve for you to be able to modify them. You don't need to commit the yarn.lock file after creating the patch.

nandorojo commented 1 year ago

I got patch-package working with Bun! Simply add --yarn when you bun install. This creates a yarn.lock file for you.

bun install --yarn

You can now use patch-package:

bunx patch-package moti
deathemperor commented 1 year ago

I found a new workaround:

  • run bun install --yarn it just converts bun.lockb into a yarn.lock

  • make the necessary modifications inside node-modules/<package-name>

  • bunx patch-package <package-name>

  • package.json -> scripts -> "postinstall": "patch-package"

    ds300/patch-package

This wont work though for platforms like vercel that determines the package manager by which lockfile you have in the repo, so if you have a yarn.lock they will use yarn to install deps unless i put in manual overrides :/

It works, my production repos are using it. Moreover, vercel already supports bun for building.

remorses commented 1 year ago

Instead of adding a bun patch command i propose adding a bun vendor package command that

This command can be used

Jarred-Sumner commented 1 year ago

we probably will do bun pm patch

we've looked into adding it but are unhappy with the performance of various diff/patch libraries

frogkind commented 11 months ago

seem patch-package not work for non-npm dependencies such as git even i manually prepare the patch file, is there any workaround?

xlc commented 11 months ago

please don't worry about performance for the first version (of course unless it is unreasonably slow). a good enough is better than not having this feature at all

rikur commented 10 months ago

+1 for having some kind of solution even if its performance is not on par with rest of the project.

oasaleh commented 10 months ago
  • unx patch-package

I followed your steps but I'm getting the following error.

bunx patch-package drizzle-kit
patch-package 8.0.0
• Creating temporary folder
• Installing drizzle-kit@0.20.13 with yarn
21 | function spawnSync(command, args, options) {
22 |     // Parse the arguments
23 |     const parsed = parse(command, args, options);
24 |
25 |     // Spawn the child process
26 |     const result = cp.spawnSync(parsed.command, parsed.args, parsed.options);
                        ^
TypeError: Executable not found in $PATH: "yarn"
 code: "ERR_INVALID_ARG_TYPE"

      at node:child_process:173:47
      at spawnSync (/private/tmp/patch-package@latest--bunx/node_modules/cross-spawn/index.js:26:20)
      at spawnSafeSync (/private/tmp/patch-package@latest--bunx/node_modules/patch-package/dist/spawnSafe.js:11:20)
      at makePatch (/private/tmp/patch-package@latest--bunx/node_modules/patch-package/dist/makePatch.js:131:17)
      at /private/tmp/patch-package@latest--bunx/node_modules/patch-package/dist/index.js:72:13
      at forEach (:1:21)
      at /private/tmp/patch-package@latest--bunx/node_modules/patch-package/dist/index.js:71:9
      at /private/tmp/patch-package@latest--bunx/node_modules/patch-package/index.js:3:1

21 | function spawnSync(command, args, options) {
22 |     // Parse the arguments
23 |     const parsed = parse(command, args, options);
24 |
25 |     // Spawn the child process
26 |     const result = cp.spawnSync(parsed.command, parsed.args, parsed.options);
                        ^
TypeError: Executable not found in $PATH: "yarn"
 code: "ERR_INVALID_ARG_TYPE"

      at node:child_process:173:47
      at spawnSync (/private/tmp/patch-package@latest--bunx/node_modules/cross-spawn/index.js:26:20)
      at spawnSafeSync (/private/tmp/patch-package@latest--bunx/node_modules/patch-package/dist/spawnSafe.js:11:20)
      at makePatch (/private/tmp/patch-package@latest--bunx/node_modules/patch-package/dist/makePatch.js:131:17)
      at /private/tmp/patch-package@latest--bunx/node_modules/patch-package/dist/index.js:72:13
      at forEach (:1:21)
      at /private/tmp/patch-package@latest--bunx/node_modules/patch-package/dist/index.js:71:9
      at /private/tmp/patch-package@latest--bunx/node_modules/patch-package/index.js:3:1

Is installing yarn a must?

corysimmons commented 10 months ago

In the meantime, I just slapped together some naive implementation of a patcher (just copies packages by name from node_modules to a ./patches dir with a command, then destructively pastes them back to node_modules with another command) to scratch this itch. It should work with anything (npm, yarn, pnpm, bun, etc.) with the same API (just swap out npx for bunx where appropriate).

https://github.com/corysimmons/patcheer

It seems to work well for my needs and should be as fast as $ cp

oasaleh commented 10 months ago

In the meantime, I just slapped together some naive implementation of a patcher (just copies packages by name from node_modules to a ./patches dir with a command, then destructively pastes them back to node_modules with another command) to scratch this itch. It should work with anything (npm, yarn, pnpm, bun, etc.) with the same API (just swap out npx for bunx where appropriate).

https://github.com/corysimmons/patcheer

It seems to work well for my needs and should be as fast as $ cp

Why can't we just edit the code in node_modules? I'm guessing it will be overwritten by the original package's code when redeployed to production. But then, how is your solution any different?

rikur commented 10 months ago

I forked a bun-friendly fork of patch-package, built it and pushed the dist/ to the repo. Works for me 🤷

Fork and build this, if you want to try the same: https://github.com/Strengthless/patch-package

corysimmons commented 10 months ago

Why can't we just edit the code in node_modules? I'm guessing it will be overwritten by the original package's code when redeployed to production. But then, how is your solution any different?

Exactly. Anytime someone (or your server when you deploy) runs bun install on your project, it is just going to install the packages straight from npmjs.com. To get around this, you commit your ./patches directory, then there are special npm script names like "postinstall" that will execute whatever is in there. So you could do "postinstall": "echo \"All my packages downloaded and installed!\"".

In most of these patching libraries, you just run the command to apply all your patches.

In my project it's "postinstall": "npx patcheer apply" (which will just forcefully/destructively/recursively copy/pastes whatever dirs are in ./patches to ./node_modules)

In patch-package (the most popular package for this kind of stuff) it's "postinstall": "patch-package" (which does a lot of complicated stuff--making it lag behind in support when these new package managers come out).

It is entirely possible my project is insanely naive and there are good reasons for a patching library to be so big, but again, my lib more than suits my needs unless someone tells me otherwise or I run into something crazy. 🤷

Update: Apparently, Bun just doesn't work on Vercel with the postinstall hook (it works locally just fine?). Since I'm working alone, I can use Bun locally (for the speed 🚗💨) and commit the Bun lockfile, and in the Vercel Dashboard I can force my build command to be npm i so postinstall works again.

On a team, or on a production project with over a few hundred users, this would be a dealbreaker as I'd want my local env to 1:1 prod as much as possible.

(per @rikur's comment immediately below... Bun builds work on Vercel if you just bun i -D postinstall-postinstall)

rikur commented 10 months ago

@corysimmons FYI I got around the postinstall issue by using postinstall-postinstall package 🙃

resthedev commented 10 months ago

@corysimmons FYI I got around the postinstall issue by using postinstall-postinstall package 🙃

@rikur Do you see Applying patches... message in your Vercel build logs? I have done the following:

But I don't see any message in my Vercel build logs to mention that the patches are being applied.

corysimmons commented 10 months ago

@resthedev I can't speak for patch-package but @rikur's fix worked for my patching package with my Vercel build. I just set my Vercel build command to bun i (I didn't append the --yarn flag), then bun add -D postinstall-postinstall & committed/pushed. And the build succeeded. 🎉

resthedev commented 10 months ago

@resthedev I can't speak for patch-package but @rikur's fix worked for my patching package with my Vercel build. I just set my Vercel build command to bun i (I didn't append the --yarn flag), then bun add -D postinstall-postinstall & committed/pushed. And the build succeeded. 🎉

I see. Do you mean you put bun i in the Vercel "build" command or the "install" command? CleanShot 2024-01-31 at 22 57 12@2x

corysimmons commented 10 months ago

Yes, it should recognize it based on there only being a bun.lockb file in your project root and automatically run bun install but I don't trust it so I just manually add bun i to it.

gustavopch commented 9 months ago

I was migrating to Bun and realized I wouldn't be able to use patch-package. I guess I'll have to wait.

corysimmons commented 9 months ago

@gustavopch I provide a workaround a couple of comments above yours.

  1. bun add -D patcheer postinstall-postinstall
  2. bunx patcheer copy <package_name>
  3. Make your tweaks in ./patches/<package_name>
  4. bunx patcheer apply
  5. Add "postinstall": "bunx patcheer apply" to your scripts in your package.json.
  6. Make your build command on your host bun install.

This should work locally and in any hosting env that supports Bun (i.e. it's working on Vercel for me). Ping me if something doesn't work.

Unsubscribing from this thread now. 💨

km-tr commented 8 months ago

@corysimmons Even if I do this, it only modifies the node_modules, and the actual running code doesn't change. But are there cases where this actually works? I'm using expo with bun.

AngeloCloudAcademy commented 8 months ago

@Jarred-Sumner some workaround in the meantime?

rikur commented 8 months ago

@AngeloCloudAcademy you can try this with postinstall-postinstall. Been using it for months now without issues.

https://github.com/oven-sh/bun/issues/2336#issuecomment-1915943659

bryanltobing commented 8 months ago

i'm doing this in the meantime

I got patch-package working with Bun! Simply add --yarn when you bun install. This creates a yarn.lock file for you.

bun install --yarn

You can now use patch-package:

bunx patch-package moti

this works, also adding yarn.lock to .gitignore so that it doesn't get commited

ReoHakase commented 8 months ago

After numerous amount of experiments, I found that patching patch-package by itself to rewrite a called package manager to bun from yarn, works. (Confirmed in local environment without yarn, GitHub Actions, Vercel)

Steps

Configure bun to generate yarn lockfile

Specify a --yarn flag like bun install --yarn or add the following lines to your bunfig.toml in order to generate yarn.lock file. This is needed since patch-package currently does not support bun's binary lockfile format. (See https://github.com/ds300/patch-package/issues/489)

[install.lockfile]
print = "yarn" # Whether to generate a non-Bun lockfile alongside bun.lockb

[!WARNING] I personally recommend the later one since you need to generate the yarn v1 lockfile every time you update dependencies.

Install patch-package

Then, add patch-package to dev dependencies.

bun add -D patch-package

Manually modify a source file of patch-package

Open node_modules/patch-package/dist/makePatch.js and look for spawnSafe_1.spawnSafeSync that spawns yarn, then replace it with bun like this diff:

diff --git a/node_modules/patch-package/dist/makePatch.js b/node_modules/patch-package/dist/makePatch.js
index d8d0925..874284a 100644
--- a/node_modules/patch-package/dist/makePatch.js
+++ b/node_modules/patch-package/dist/makePatch.js
@@ -120,7 +120,7 @@ function makePatch({ packagePathSpecifier, appPath, packageManager, includePaths
             try {
                 // try first without ignoring scripts in case they are required
                 // this works in 99.99% of cases
-                spawnSafe_1.spawnSafeSync(`yarn`, ["install", "--ignore-engines"], {
+                spawnSafe_1.spawnSafeSync(`bun`, ["install"], {
                     cwd: tmpRepoNpmRoot,
                     logStdErrOnError: false,
                 });
@@ -128,7 +128,7 @@ function makePatch({ packagePathSpecifier, appPath, packageManager, includePaths
             catch (e) {
                 // try again while ignoring scripts in case the script depends on
                 // an implicit context which we haven't reproduced
-                spawnSafe_1.spawnSafeSync(`yarn`, ["install", "--ignore-engines", "--ignore-scripts"], {
+                spawnSafe_1.spawnSafeSync(`bun`, ["install", "--ignore-scripts"], {
                     cwd: tmpRepoNpmRoot,
                 });
             }

[!NOTE] You can remove --ignore-engines flag without worrying since bun does not support engines field in package.json at the moment. (See https://github.com/oven-sh/bun/issues/5846)

Save the patch you've made to patches/

This command looks like pure madness, but surprisingly it worked 😮

bun patch-package patch-package

Now, patches/patch-package+8.0.0.patch should be created.

Add postinstall command to package.json

As it is written in patch-package documentation.

{
  "name": "reoiam-dev",
  "version": "0.0.0",
  "workspaces": [
    "apps/*",
    "packages/*"
  ],
  "scripts": {
    "postinstall": "patch-package",
  },

Modify source files of your desired package

bun patch-package @acme/some-package

Also, run git clean -dfX && bun i to check if it is working.

Example commits: https://github.com/ReoHakase/reoiam-dev/pull/122/commits/2c016376918356d46ae59a2199e687aec085fb76 https://github.com/ReoHakase/reoiam-dev/pull/122/commits/8aae34e0c072b389b165a04870ba9a631355d1a6

rChaoz commented 7 months ago

I would also like to add that it would be awesome to support something like Yarn does with resolutions:

package.json:

{
    "resolutions": {
        "some-library:1.2.3": "patch:patch-file",
        "other-library:1.0.0": "http://alternate-source/file.tgz",
        "final-library:100.0.0": "./dir/final-library-fork"
    }
}

This support pretty much every use case:

  1. simple bugfixes can be made directly on the modules using a patch tool
  2. more complex changes can be made by forking a library repo, cloning, changing & building it locally
  3. sometimes, the process from 2 is simplified by CI tools that provide a release version for PRs, so including the library clone in your code is not needed
vijaybritto commented 6 months ago

To make it work with Private registries

I followed the steps by @ReoHakase and it worked for public packages. But to make it work with our private registry I had to do the following changes:

diff --git a/node_modules/patch-package/dist/makePatch.js b/node_modules/patch-package/dist/makePatch.js
index d8d0925..7c488cd 100644
--- a/node_modules/patch-package/dist/makePatch.js
+++ b/node_modules/patch-package/dist/makePatch.js
@@ -109,18 +109,18 @@ function makePatch({ packagePathSpecifier, appPath, packageManager, includePaths
             resolutions: resolveRelativeFileDependencies_1.resolveRelativeFileDependencies(appPath, appPackageJson.resolutions || {}),
         }));
         const packageVersion = getPackageVersion_1.getPackageVersion(path_1.join(path_1.resolve(packageDetails.path), "package.json"));
-        [".npmrc", ".yarnrc", ".yarn"].forEach((rcFile) => {
+        [".npmrc", ".yarnrc", ".yarn", "bunfig.toml", ".env"].forEach((rcFile) => {
             const rcPath = path_1.join(appPath, rcFile);
             if (fs_extra_1.existsSync(rcPath)) {
                 fs_extra_1.copySync(rcPath, path_1.join(tmpRepo.name, rcFile), { dereference: true });
             }
         });
         if (packageManager === "yarn") {
-            console_1.default.info(chalk_1.default.grey("•"), `Installing ${packageDetails.name}@${packageVersion} with yarn`);
+            console_1.default.info(chalk_1.default.grey("•"), `Installing ${packageDetails.name}@${packageVersion} with bun`);
             try {
                 // try first without ignoring scripts in case they are required
                 // this works in 99.99% of cases
-                spawnSafe_1.spawnSafeSync(`yarn`, ["install", "--ignore-engines"], {
+                spawnSafe_1.spawnSafeSync(`bun`, ["install"], {
                     cwd: tmpRepoNpmRoot,
                     logStdErrOnError: false,
                 });
@@ -128,7 +128,7 @@ function makePatch({ packagePathSpecifier, appPath, packageManager, includePaths
             catch (e) {
                 // try again while ignoring scripts in case the script depends on
                 // an implicit context which we haven't reproduced
-                spawnSafe_1.spawnSafeSync(`yarn`, ["install", "--ignore-engines", "--ignore-scripts"], {
+                spawnSafe_1.spawnSafeSync(`bun`, ["install", "--ignore-scripts"], {
                     cwd: tmpRepoNpmRoot,
                 });
             }

I had to specify "bunfig.toml", ".env" extra in the list of files to copy into the temporary folder. This is to make sure that bun can find the private registry in the bunfig.toml file.

To Summarize:

  1. Change bunfig.toml to add the following 
    [install.lockfile]
print = "yarn" # Whether to generate a non-Bun lockfile alongside bun.lockb
  2. Add the package 
    bun add -D patch-package
  3. Make changes in node_modules/patch-package/dist/makePatch.js as shown above
  4. Create the patch for patch-package 
    bun patch-package patch-package
  5. Add postinstall script 
    {
"name": "reoiam-dev",
"version": "0.0.0",
"workspaces": [
"apps/*",
"packages/*"
],
"scripts": {
"postinstall": "patch-package",
}
  6. Optionally verify it by forcing a full reinstall 
    bun install --force
  7. Follow regular patch-package flows
zackradisic commented 6 months ago

For those tracking this issue, I am currently working on this. Patching a package will behave similarly to pnpm, through the bun patch <pkg> and bun patch-commit <path> commands.

Yarn's patch protocol will also be supported.

My work-in-progress PR for this feature is here.

Jarred-Sumner commented 5 months ago

The merged implementation of bun patch does not meet our quality bar. Leaving this open until #11719 is completed

Jarred-Sumner commented 5 months ago

Fixed by @zackradisic in #11858

Scalahansolo commented 5 months ago

Attempted to use this and am running into this issue? Am I doing something wrong here?

CleanShot 2024-06-19 at 22 03 31@2x

zackradisic commented 5 months ago

@Scalahansolo My fixes in this PR (#12022) might fix this. There were issues running bun patch in workspaces and with scoped packages

Scalahansolo commented 5 months ago

Still doesn't work in our monorepo on 1.1.16

cc @zackradisic

CleanShot 2024-06-23 at 21 23 26@2x

Jarred-Sumner commented 5 months ago

@Scalahansolo can you file an issue and @zackradisic will fix it?

nobkd commented 5 months ago

Is there a way to prepare multiple packages for patching at once? Something like bun patch pkg1 pkg2 and same for bun patch --commit pkg1 pkg2