search

jaredpalmer / tsdx

Zero-config CLI for TypeScript package development
https://tsdx.io
MIT License
11.24k stars 508 forks source link

v0.14.0 rpts2 cache issue when building `umd` with `cjs` #892

Closed MagnusBrzenk closed 3 years ago

MagnusBrzenk commented 3 years ago

Current Behavior

Right out of the box, running tsdx build --format umd,cjs,esm causes following error:

> tsdx build --format esm,cjs,umd

✓ Creating entry file 1.1 secs
(typescript) Error: ENOENT: no such file or directory, rename '/Users/magnus/work/javascript/myapp/node_modules/.cache/rollup-plugin-typescript2/rpt2_4c61ae4392b9bd24f4d43d13a3f56419b8b5805d/code/cache_' -> '/Users/magnus/work/javascript/myapp/node_modules/.cache/rollup-plugin-typescript2/rpt2_4c61ae4392b9bd24f4d43d13a3f56419b8b5805d/code/cache'
Error: ENOENT: no such file or directory, rename '/Users/magnus/work/javascript/myapp/node_modules/.cache/rollup-plugin-typescript2/rpt2_4c61ae4392b9bd24f4d43d13a3f56419b8b5805d/code/cache_' -> '/Users/magnus/work/javascript/myapp/node_modules/.cache/rollup-plugin-typescript2/rpt2_4c61ae4392b9bd24f4d43d13a3f56419b8b5805d/code/cache'
    at Object.renameSync (fs.js:660:3)
    at RollingCache.roll (/Users/magnus/work/javascript/myapp/node_modules/tsdx/node_modules/rollup-plugin-typescript2/dist/rollup-plugin-typescript2.cjs.js:24003:12)
    at TsCache.done (/Users/magnus/work/javascript/myapp/node_modules/tsdx/node_modules/rollup-plugin-typescript2/dist/rollup-plugin-typescript2.cjs.js:24845:24)
    at Object._ongenerate (/Users/magnus/work/javascript/myapp/node_modules/tsdx/node_modules/rollup-plugin-typescript2/dist/rollup-plugin-typescript2.cjs.js:29098:21)
    at Object.generateBundle (/Users/magnus/work/javascript/myapp/node_modules/tsdx/node_modules/rollup-plugin-typescript2/dist/rollup-plugin-typescript2.cjs.js:29076:18)
    at /Users/magnus/work/javascript/myapp/node_modules/rollup/dist/shared/node-entry.js:13117:25
    at processTicksAndRejections (internal/process/task_queues.js:97:5)

npm ERR! code ELIFECYCLE
npm ERR! errno 1
npm ERR! myapp@0.1.0 build: `tsdx build --format esm,cjs,umd`
npm ERR! Exit status 1
npm ERR!
npm ERR! Failed at the myapp@0.1.0 build script.
npm ERR! This is probably not a problem with npm. There is likely additional logging output above.

npm ERR! A complete log of this run can be found in:
npm ERR!     /Users/magnus/.npm/_logs/2020-10-01T16_45_40_976Z-debug.log

After trying different combinations, I discerned that 'umd' is incompatible with 'cjs'.

Expected behavior

The documentation gives the impression that you can build all three targets simultaneously by running tsdx build --format umd,cjs,esm. If this is not possible to build both 'umd' and 'cjs' types at once then clarify this is in the documentation; otherwise you have a bug here.

Suggested solution(s)

Fix either documentation or underlying problem.

Additional context

Your environment

  System:
    OS: macOS 10.15.6
    CPU: (8) x64 Intel(R) Core(TM) i7-7700HQ CPU @ 2.80GHz
    Memory: 341.14 MB / 16.00 GB
    Shell: 5.7.1 - /bin/zsh
  Binaries:
    Node: 12.16.1 - ~/.nvm/versions/node/v12.16.1/bin/node
    Yarn: 1.22.5 - ~/.nvm/versions/node/v12.16.1/bin/yarn
    npm: 6.13.4 - ~/.nvm/versions/node/v12.16.1/bin/npm
  Browsers:
    Brave Browser: 85.1.14.81
    Chrome: 85.0.4183.121
    Firefox: 81.0
    Safari: 14.0
  npmPackages:
    tsdx: ^0.14.0 => 0.14.0
    typescript: ^4.0.3 => 4.0.3
agilgur5 commented 3 years ago

Based on the logs, sounds like a duplicate of #888. (EDIT: I've now marked that issue as duplicative of this issue because this one has more details now.) Per my response there:

Looks like this is a cache bug upstream that was recently fixed: https://github.com/ezolenko/rollup-plugin-typescript2/pull/243

Per comments there, you can workaround this by deleting the cache. I don't think you need to fully nuke node_modules, think you can just delete node_modules/.cache/rollup-plugin-typescript2/

After trying different combinations, I discerned that 'umd' is incompatible with 'cjs'.

That's an interesting find though upstream in rollup-plugin-typescript2 it has no knowledge of format (I've worked on that codebase several times now), so more likely it's some bad hash and race condition that happens during updating

The documentation gives the impression that you can build all three targets simultaneously

You can, a central piece of TSDX and even Rollup itself, and there are tests for this too. You just appear to have hit a transient cache bug upstream, which the logs point to.

agilgur5 commented 3 years ago

So I tried a few things and confirmed the umd + cjs issue. I also confirmed it was happening with system + cjs. But not esm + cjs for some reason. I also tried ordering it differently, but that didn't change anything (they are parallelized in any case but 🤷 ). I also confirmed umd + system had the issue as well.

The system addition is interesting, as that shouldn't have much difference from the others aside from format. system + umd also knocks the suggestion that it's some cjs issue out 😕 😕

Here's some logs: ```bash $$ tsdx create test-tsdx && cd test-tsdx [...] $$ yarn build --format umd yarn run v1.22.4 $ tsdx build --format umd ✓ Building modules 674 ms ✨ Done in 3.96s. $$ yarn build --format cjs,umd yarn run v1.22.4 $ tsdx build --format cjs,umd ✓ Creating entry file 967 ms (typescript) Error: ENOENT: no such file or directory, rename '[redacted]/test-tsdx/node_modules/.cache/rollup-plugin-typescript2/rpt2_d8917360bd7c6c37111985035e75a4930df504e4/code/cache_' - > '[redacted]/test-tsdx/node_modules/.cache/rollup-plugin-typescript2/rpt2_d8917360bd7c6c37111985035e75a4930df504e4/code/cache' [...] $$ yarn build --format system,umd yarn run v1.22.4 $ tsdx build --format system,umd (typescript) Error: ENOENT: no such file or directory, rename '[redacted]/test-tsdx/node_modules/.cache/rollup-plugin-typescript2/rpt2_d8917360bd7c6c37111985035e75a4930df504e4/code/cache_' - > '[redacted]/test-tsdx/node_modules/.cache/rollup-plugin-typescript2/rpt2_d8917360bd7c6c37111985035e75a4930df504e4/code/cache' [...] $$ yarn build --format cjs,esm yarn run v1.22.4 $ tsdx build --format cjs,esm ✓ Creating entry file 957 ms ✓ Building modules 1.1 secs ✨ Done in 3.86s. $$ yarn build --format cjs,esm,umd yarn run v1.22.4 $ tsdx build --format cjs,esm,umd ✓ Creating entry file 954 ms (typescript) Error: ENOENT: no such file or directory, rename '[redacted]/test-tsdx/node_modules/.cache/rollup-plugin-typescript2/rpt2_d8917360bd7c6c37111985035e75a4930df504e4/code/cache_' - > '[redacted]/test-tsdx/node_modules/.cache/rollup-plugin-typescript2/rpt2_d8917360bd7c6c37111985035e75a4930df504e4/code/cache' [...] $$ yarn build --format cjs,esm,system yarn run v1.22.4 $ tsdx build --format cjs,esm,system ✓ Creating entry file 963 ms (typescript) Error: ENOENT: no such file or directory, rename '[redacted]/test-tsdx/node_modules/.cache/rollup-plugin-typescript2/rpt2_d8917360bd7c6c37111985035e75a4930df504e4/code/cache_' - > '[redacted]/test-tsdx/node_modules/.cache/rollup-plugin-typescript2/rpt2_d8917360bd7c6c37111985035e75a4930df504e4/code/cache' [...] $$ yarn build --format cjs,system yarn run v1.22.4 $ tsdx build --format cjs,system ✓ Creating entry file 969 ms (typescript) Error: ENOENT: no such file or directory, rename '[redacted]/test-tsdx/node_modules/.cache/rollup-plugin-typescript2/rpt2_d8917360bd7c6c37111985035e75a4930df504e4/code/cache_' - > '[redacted]/test-tsdx/node_modules/.cache/rollup-plugin-typescript2/rpt2_d8917360bd7c6c37111985035e75a4930df504e4/code/cache' $$ yarn build --format esm,system yarn run v1.22.4 $ tsdx build --format esm,system ✓ Building modules 1.2 secs ✨ Done in 3.94s. $$ yarn build --format esm,umd yarn run v1.22.4 $ tsdx build --format esm,umd ✓ Building modules 634 ms ✨ Done in 3.36s. # different ordering $$ yarn build --format esm,cjs,umd yarn run v1.22.4 $ tsdx build --format esm,cjs,umd ✓ Creating entry file 980 ms (typescript) Error: ENOENT: no such file or directory, rename '[redacted]/test-tsdx/node_modules/.cache/rollup-plugin-typescript2/rpt2_d8917360bd7c6c37111985035e75a4930df504e4/code/cache_' - > '[redacted]/test-tsdx/node_modules/.cache/rollup-plugin-typescript2/rpt2_d8917360bd7c6c37111985035e75a4930df504e4/code/cache' [...] # different ordering $$ yarn build --format esm,umd,cjs yarn run v1.22.4 $ tsdx build --format esm,umd,cjs ✓ Creating entry file 979 ms (typescript) Error: ENOENT: no such file or directory, rename '[redacted]/test-tsdx/node_modules/.cache/rollup-plugin-typescript2/rpt2_d8917360bd7c6c37111985035e75a4930df504e4/code/cache_' - > '[redacted]/test-tsdx/node_modules/.cache/rollup-plugin-typescript2/rpt2_d8917360bd7c6c37111985035e75a4930df504e4/code/cache' [...] $$ yarn build --format esm,umd,system yarn run v1.22.4 $ tsdx build --format esm,umd,system (typescript) Error: ENOENT: no such file or directory, rename '[redacted]/test-tsdx/node_modules/.cache/rollup-plugin-typescript2/rpt2_dc5aaa4ea403c2ba50e68d0baecc90e56a88ad12/code/cache_' - > '[redacted]/test-tsdx/node_modules/.cache/rollup-plugin-typescript2/rpt2_dc5aaa4ea403c2ba50e68d0baecc90e56a88ad12/code/cache' [...] $$ yarn build --format umd,system yarn run v1.22.4 $ tsdx build --format umd,system (typescript) Error: ENOENT: no such file or directory, rename '[redacted]/test-tsdx/node_modules/.cache/rollup-plugin-typescript2/rpt2_d8917360bd7c6c37111985035e75a4930df504e4/code/cache_' - > '[redacted]/test-tsdx/node_modules/.cache/rollup-plugin-typescript2/rpt2_d8917360bd7c6c37111985035e75a4930df504e4/code/cache' [...] $$ yarn build --format umd,cjs yarn run v1.22.4 $ tsdx build --format umd,cjs ✓ Creating entry file 962 ms (typescript) Error: ENOENT: no such file or directory, rename '[redacted]/oss/test-tsdx/node_modules/.cache/rollup-plugin-typescript2/rpt2_d8917360bd7c6c37111985035e75a4930df504e4/code/cache_' - > '[redacted]/oss/test-tsdx/node_modules/.cache/rollup-plugin-typescript2/rpt2_d8917360bd7c6c37111985035e75a4930df504e4/code/cache' ```
agilgur5 commented 3 years ago

Additionally confirmed that this issue did exist in TSDX v0.13.3, but only for the cjs + system combo, which is much less frequent. Was confused that it hadn't happened with v0.13.3 despite it using the same version of rpts2 as v0.14.0, but this allays some of that. And also makes it seem more of a hashing bug or race condition upstream than a specific format, which would've been confusing too given the lack of knowledge of format.

More logs: ```bash $ yarn add tsdx@0.13.3 yarn add v1.22.4 [...] $ tsdx build ✓ Creating entry file 1.3 secs ✓ Building modules 2.2 secs ✨ Done in 17.88s. $$ yarn build --format umd,cjs yarn run v1.22.4 $ tsdx build --format umd,cjs ✓ Creating entry file 1 secs ✓ Building modules 1.9 secs ✨ Done in 4.10s. $$ yarn build --format umd,cjs,esm,system yarn run v1.22.4 $ tsdx build --format umd,cjs,esm,system ✓ Creating entry file 919 ms (typescript) Error: ENOENT: no such file or directory, rename '[redacted]/test-tsdx/node_modules/.cache/rollup-plugin-typescript2/rpt2_57678ef6f24a79cbc0a37de5d02872e29b66702a/code/cache_' - > '[redacted]/test-tsdx/node_modules/.cache/rollup-plugin-typescript2/rpt2_57678ef6f24a79cbc0a37de5d02872e29b66702a/code/cache' [...] $$ yarn build --format umd,esm yarn run v1.22.4 $ tsdx build --format umd,esm ✓ Building modules 715 ms ✨ Done in 2.68s. $$ yarn build --format umd,esm,cjs yarn run v1.22.4 $ tsdx build --format umd,esm,cjs ✓ Creating entry file 924 ms ✓ Building modules 1.4 secs ✨ Done in 3.40s. $$ yarn build --format umd,system yarn run v1.22.4 $ tsdx build --format umd,system ✓ Building modules 907 ms ✨ Done in 2.86s. ✔ ~/Desktop/GitHub/oss/test-tsdx (⎈ |services.k8s.dev.ns8.ninja:default) $$ yarn build --format cjs,system yarn run v1.22.4 $ tsdx build --format cjs,system ✓ Creating entry file 915 ms (typescript) Error: ENOENT: no such file or directory, rename '[redacted]/test-tsdx/node_modules/.cache/rollup-plugin-typescript2/rpt2_57678ef6f24a79cbc0a37de5d02872e29b66702a/code/cache_' - > '[redacted]/test-tsdx/node_modules/.cache/rollup-plugin-typescript2/rpt2_57678ef6f24a79cbc0a37de5d02872e29b66702a/code/cache' [...] $$ yarn build --format cjs,esm,umd yarn run v1.22.4 $ tsdx build --format cjs,esm,umd ✓ Creating entry file 924 ms ✓ Building modules 1.4 secs ✨ Done in 3.43s. $$ yarn build --format umd,system yarn run v1.22.4 $ tsdx build --format umd,system ✓ Building modules 897 ms ✨ Done in 2.87s. $$ yarn build --format cjs,system yarn run v1.22.4 $ tsdx build --format cjs,system ✓ Creating entry file 913 ms (typescript) Error: ENOENT: no such file or directory, rename '[redacted]/test-tsdx/node_modules/.cache/rollup-plugin-typescript2/rpt2_57678ef6f24a79cbc0a37de5d02872e29b66702a/code/cache_' - > '[redacted]/test-tsdx/node_modules/.cache/rollup-plugin-typescript2/rpt2_57678ef6f24a79cbc0a37de5d02872e29b66702a/code/cache' [...] $$ yarn build --format cjs,esm,umd yarn run v1.22.4 $ tsdx build --format cjs,esm,umd ✓ Creating entry file 914 ms ✓ Building modules 1.4 secs ✨ Done in 3.34s. ```
agilgur5 commented 3 years ago

Hmmm v0.13.2, v0.13.1, v0.13.0, and v0.12 don't have this issue 🤔 That being said, I'm not sure I'd call this a regression per se, because, per my investigation in #896 and https://github.com/ezolenko/rollup-plugin-typescript2/pull/243, that section of code upstream has been like that for a long time. But TSDX's cache directories and Rollup config (which impacts the hash) have changed over time. 🤔

Side note: the perf decrease when downgrading was noticeable, so that confirms the more recent optimizations I've made help in a variety of codebase sizes 🙂

v0.13.3 consolidated the cache directories for performance reasons (more matching hashes); previously there was a different one for each format, so perhaps that creates more races or makes for an incorrectly matched hash. The problematic cache seems to have an identical hash in all of the logs (only one is different from the rest) 🤔

Logs: ```bash $$ yarn add tsdx@0.13.2 yarn add v1.22.4 [...] $ tsdx build ✓ Creating entry file 1.8 secs ✓ Building modules 753 ms ✨ Done in 7.66s. $$ yarn build --format cjs,esm,umd,system yarn run v1.22.4 $ tsdx build --format cjs,esm,umd,system ✓ Creating entry file 3.4 secs ✓ Building modules 1.3 secs ✨ Done in 5.88s. $$ yarn build --format umd,system yarn run v1.22.4 $ tsdx build --format umd,system ✓ Building modules 944 ms ✨ Done in 4.15s. $$ yarn build --format umd,cjs,esm yarn run v1.22.4 $ tsdx build --format umd,cjs,esm ✓ Creating entry file 2.6 secs ✓ Building modules 912 ms ✨ Done in 4.65s. $$ yarn build --format umd,cjs yarn run v1.22.4 $ tsdx build --format umd,cjs ✓ Creating entry file 2.1 secs ✓ Building modules 985 ms ✨ Done in 4.18s. $$ yarn build --format umd,esm yarn run v1.22.4 $ tsdx build --format umd,esm ✓ Building modules 568 ms ✨ Done in 3.46s. $$ yarn build --format cjs,system yarn run v1.22.4 $ tsdx build --format cjs,system ✓ Creating entry file 2.2 secs ✓ Building modules 939 ms ✨ Done in 4.22s. $$ yarn build --format esm,system yarn run v1.22.4 $ tsdx build --format esm,system ✓ Building modules 618 ms ✨ Done in 3.43s. $$ yarn build --format esm,umd,system yarn run v1.22.4 $ tsdx build --format esm,umd,system ✓ Building modules 1.1 secs ✨ Done in 4.93s. $$ yarn build --format cjs,umd,system yarn run v1.22.4 $ tsdx build --format cjs,umd,system ✓ Creating entry file 3 secs ✓ Building modules 1.2 secs ✨ Done in 5.25s. $$ yarn build --format cjs,esm,umd yarn run v1.22.4 $ tsdx build --format cjs,esm,umd ✓ Creating entry file 2.6 secs ✓ Building modules 916 ms ✨ Done in 4.63s. $$ yarn add tsdx@0.13.1 yarn add v1.22.4 [...] $ tsdx build ✓ Creating entry file 1.8 secs ✓ Building modules 686 ms ✨ Done in 8.09s. $$ yarn build --format cjs,esm,umd,system yarn run v1.22.4 $ tsdx build --format cjs,esm,umd,system ✓ Creating entry file 3.5 secs ✓ Building modules 1.3 secs ✨ Done in 5.91s. # second time for cache etc $$ yarn build --format cjs,esm,umd,system yarn run v1.22.4 $ tsdx build --format cjs,esm,umd,system ✓ Creating entry file 3.5 secs ✓ Building modules 1.3 secs ✨ Done in 5.87s. $$ yarn add tsdx@0.13.0 yarn add v1.22.4 [...] $ tsdx build ✓ Creating entry file 1.9 secs ✓ Building modules 652 ms ✨ Done in 7.44s. $$ yarn build --format cjs,esm,umd,system yarn run v1.22.4 $ tsdx build --format cjs,esm,umd,system ✓ Creating entry file 3.4 secs ✓ Building modules 1.2 secs ✨ Done in 5.75s. # second time for cache etc $$ yarn build --format cjs,esm,umd,system yarn run v1.22.4 $ tsdx build --format cjs,esm,umd,system ✓ Creating entry file 3.5 secs ✓ Building modules 1.2 secs ✨ Done in 5.76s. $$ yarn add tsdx@0.12 [...] $ tsdx build ✓ Creating entry file 2.4 secs ✓ Building modules 960 ms ✨ Done in 9.50s. $$ yarn build --format cjs,esm,umd,system yarn run v1.22.4 $ tsdx build --format cjs,esm,umd,system ✓ Creating entry file 3.7 secs ✓ Building modules 1.4 secs ✨ Done in 6.28s. # second time for cache etc $$ yarn build --format cjs,esm,umd,system yarn run v1.22.4 $ tsdx build --format cjs,esm,umd,system ✓ Creating entry file 3.7 secs ✓ Building modules 1.3 secs ✨ Done in 6.14s. ```
agilgur5 commented 3 years ago

This has been fixed with #896 / https://github.com/ezolenko/rollup-plugin-typescript2/pull/243 and just released in v0.14.1.

Still waiting for a response upstream to understand root cause however.