search

oven-sh / bun

Incredibly fast JavaScript runtime, bundler, test runner, and package manager – all in one
https://bun.sh
Other
73.16k stars 2.68k forks source link

`bun install` routinely accumulates package duplication #13533

Open brettwillis opened 3 weeks ago

brettwillis commented 3 weeks ago

What version of Bun is running?

1.1.26

What platform is your computer?

Darwin 23.6.0 arm64 arm

What steps can reproduce the bug?

When updating package versions, either manually in the package.json and running bun install, or by running bun update, Bun routinely starts duplicating packages in the dependency tree and then continues to accumulate duplicated dependencies over time, entirely un-necessarily.

This happens silently until we realise what has been happening due to obscure runtime bugs or TypeScript type checking errors.

Now, we always need to blast away the entire installation and lockfile and install from scratch every time to avoid these obscure problems.

Reproduction:

Consider then packages

When installing, we get a good tree like this:

% bun pm ls --all
./myproject/core node_modules
├── @myorg/dep1@1.0.89
├── @myorg/dep2@1.0.89

Now, @myorg/dep1 is updated to 1.0.90, and we update the dependency, and the tree becomes:

% bun pm ls --all
./myproject/core node_modules
├── @myorg/dep1@1.0.90
├── @myorg/dep2@1.0.89
│   └── @myorg/dep1@1.0.89

This prolific duplication seems crazy.

What is the expected behavior?

I expect Bun should only duplicate the dependency tree when absolutely necessary to maintain semver compatibility. Dependencies should always be deduplicated/hoisted except where the hoisted version cannot meet semver requirements.

TL;DR prioritise having a single version of each dependency.

What do you see instead?

See above.

Additional information

npm update

npm will prioritize having a single version of [dependencies] in your tree rather than two when that single version can satisfy the semver requirements of multiple dependencies in your tree.

npm install

hoisted (default): Install non-duplicated in top-level, and duplicated as necessary within directory structure

Jarred-Sumner commented 3 weeks ago

Can you paste a package.json + bun.lockb with a more clear example?

brettwillis commented 3 weeks ago

Sure, note these are private org dependencies, but hopefully the info can tell you what you need.

Step 1

package.json

{
  "dependencies": {
    "@arrowheadapps/billing-engine-core": "1.0.89",
    "@arrowheadapps/billing-engine-platform-core": "1.0.89"
  }
}

bun.lockb

# THIS IS AN AUTOGENERATED FILE. DO NOT EDIT THIS FILE DIRECTLY.
# yarn lockfile v1
# bun ./bun.lockb --hash: 4677EDE48AFE075F-9887327d4f9c38db-9FB4A885D13A430A-aba16b62138a868f

"@arrowheadapps/billing-engine-core@1.0.89", "@arrowheadapps/billing-engine-core@^1.0.88": 
  version "1.0.89"
  resolved "https://npm.pkg.github.com/download/@arrowheadapps/billing-engine-core/1.0.89/b7f345abb57b4ba15cc5a6ef475f2b85d01ad0f4"
  integrity sha512-24l6Z/5s18UdHSVysjmjFitFalfBdXGM7VQ9WchImq6Bh3/q/UDjDp/1fBmRrplsL1cytKbVYBHDpkQxURKdaw==
  dependencies: 
    date-fns "^3.6.0"
    date-fns-tz "^3.1.3"

"@arrowheadapps/billing-engine-platform-core@1.0.89": 
  version "1.0.89"
  resolved "https://npm.pkg.github.com/download/@arrowheadapps/billing-engine-platform-core/1.0.89/c98cfd0745e018887356f4b33a84f9dd7da51369"
  integrity sha512-5Cv+VWHJI951sCIiPQs1rYww8VEbv7LrU//CtFivTEMq/WWEgXzmJSMkbxRQCPs8mD16BVgG6cpLyUpt1nYUeA==
  dependencies: 
    "@arrowheadapps/billing-engine-core" "^1.0.88"
    "@arrowheadapps/database" "^1.24.0"

"@arrowheadapps/database@^1.24.0": 
  version "1.24.0"
  resolved "https://npm.pkg.github.com/download/@arrowheadapps/database/1.24.0/d8c5e785abada06c63015a71ac75477099564e06"
  integrity sha512-rfK4rT5SyDBZVly/8xrAnekHsGFbrhNRYoOi+Ehe1BEm9D3iMS3aQBD+zKZz3GaxcimqhSKdMJCezZpwkKvQLA==

date-fns@^3.0.0, date-fns@^3.6.0: 
  version "3.6.0"
  resolved "https://registry.npmjs.org/date-fns/-/date-fns-3.6.0.tgz"
  integrity sha512-fRHTG8g/Gif+kSh50gaGEdToemgfj74aRX3swtiouboip5JDLAyDE9F11nHMIcvOaXeOC6D7SpNhi7uFyB7Uww==

date-fns-tz@^3.1.3: 
  version "3.1.3"
  resolved "https://registry.npmjs.org/date-fns-tz/-/date-fns-tz-3.1.3.tgz"
  integrity sha512-ZfbMu+nbzW0mEzC8VZrLiSWvUIaI3aRHeq33mTe7Y38UctKukgqPR4nTDwcwS4d64Gf8GghnVsroBuMY3eiTeA==

Step 2

Changed versions in package.json to allow upgrading, ran bun update.

package.json

{
  "dependencies": {
    "@arrowheadapps/billing-engine-core": "^1.0.90",
    "@arrowheadapps/billing-engine-platform-core": "^1.0.89"
  }
}

Edit: sorry I originally didn't refresh the view of the lockfile, updated with the correct content:

bun.lockb

# THIS IS AN AUTOGENERATED FILE. DO NOT EDIT THIS FILE DIRECTLY.
# yarn lockfile v1
# bun ./bun.lockb --hash: BAEA39DBE85E244A-590f4688d801b0ed-46555B4DF4C22FF4-aa6ebccac756b473

"@arrowheadapps/billing-engine-core@^1.0.88": 
  version "1.0.89"
  resolved "https://npm.pkg.github.com/download/@arrowheadapps/billing-engine-core/1.0.89/b7f345abb57b4ba15cc5a6ef475f2b85d01ad0f4"
  integrity sha512-24l6Z/5s18UdHSVysjmjFitFalfBdXGM7VQ9WchImq6Bh3/q/UDjDp/1fBmRrplsL1cytKbVYBHDpkQxURKdaw==
  dependencies: 
    date-fns "^3.6.0"
    date-fns-tz "^3.1.3"

"@arrowheadapps/billing-engine-core@^1.0.89": 
  version "1.0.90"
  resolved "https://npm.pkg.github.com/download/@arrowheadapps/billing-engine-core/1.0.90/32e97822444abfc3a479959d7ce21460a75a0899"
  integrity sha512-hkBFsAzN1yHyKRPN3yYDqypoFKuslomTdaNy3ixaXRQ8nHIgGgARa3jZZhrx2a+LY97mKoQYNF+OVZgO1bnjZg==
  dependencies: 
    date-fns "^3.6.0"
    date-fns-tz "^3.1.3"

"@arrowheadapps/billing-engine-platform-core@^1.0.89": 
  version "1.0.89"
  resolved "https://npm.pkg.github.com/download/@arrowheadapps/billing-engine-platform-core/1.0.89/c98cfd0745e018887356f4b33a84f9dd7da51369"
  integrity sha512-5Cv+VWHJI951sCIiPQs1rYww8VEbv7LrU//CtFivTEMq/WWEgXzmJSMkbxRQCPs8mD16BVgG6cpLyUpt1nYUeA==
  dependencies: 
    "@arrowheadapps/billing-engine-core" "^1.0.88"
    "@arrowheadapps/database" "^1.24.0"

"@arrowheadapps/database@^1.24.0": 
  version "1.24.0"
  resolved "https://npm.pkg.github.com/download/@arrowheadapps/database/1.24.0/d8c5e785abada06c63015a71ac75477099564e06"
  integrity sha512-rfK4rT5SyDBZVly/8xrAnekHsGFbrhNRYoOi+Ehe1BEm9D3iMS3aQBD+zKZz3GaxcimqhSKdMJCezZpwkKvQLA==

date-fns@^3.0.0, date-fns@^3.6.0: 
  version "3.6.0"
  resolved "https://registry.npmjs.org/date-fns/-/date-fns-3.6.0.tgz"
  integrity sha512-fRHTG8g/Gif+kSh50gaGEdToemgfj74aRX3swtiouboip5JDLAyDE9F11nHMIcvOaXeOC6D7SpNhi7uFyB7Uww==

date-fns-tz@^3.1.3: 
  version "3.1.3"
  resolved "https://registry.npmjs.org/date-fns-tz/-/date-fns-tz-3.1.3.tgz"
  integrity sha512-ZfbMu+nbzW0mEzC8VZrLiSWvUIaI3aRHeq33mTe7Y38UctKukgqPR4nTDwcwS4d64Gf8GghnVsroBuMY3eiTeA==

And output of bun pm ls --all (yes it does correctly reflect the filesystem):

% bun pm ls --all
./test node_modules
├── @arrowheadapps/billing-engine-core@1.0.90
├── @arrowheadapps/billing-engine-platform-core@1.0.89
│   └── @arrowheadapps/billing-engine-core@1.0.89
├── @arrowheadapps/database@1.24.0
├── date-fns@3.6.0
└── date-fns-tz@3.1.3
Jarred-Sumner commented 3 weeks ago

Thanks. @dylan-conway is able to reproduce it