yarnpkg / berry

📦🐈 Active development trunk for Yarn ⚒
https://yarnpkg.com
BSD 2-Clause "Simplified" License
7.41k stars 1.11k forks source link

[Feature] Add config option to store packages in non zipped folder #1894

Closed remorses closed 4 years ago

remorses commented 4 years ago

Describe the user story

If packages were stored in a simple folder without zipping them it would be much easier to use IDE features like go to reference and reading stack traces

From what I understand the only reason we are using zipFS is to make dependencies on disk smaller, I would happily opt out of that feature to get better stack traces and debugging capabilities (for example adding some console.log in a package to debug some dependency)

Describe the solution you'd like

Store packages in a folder without zipping them

Describe the drawbacks of your solution

Slightly larger packages size on disk

I don’t really know how yarn pnp works under the hood, I hope this feature request makes sense

Describe alternatives you've considered

Additional context

arcanis commented 4 years ago

the only reason we are using zipFS is to make dependencies on disk smaller

That's not the case - it's also for sanity, as the package manager can then assume various extra things (that the archive checksum will change if the package content is corrupted in any way, that the package content will be exactly the same, to the chmod and mtime, regardless of the system, that installs are fast, etc).

You can opt-out on a per-package basis by using yarn unplug (and using * should unplug everything, although I wouldn't recommend it), that should be enough (we have plans to improve the yarn unplug DX to not change the files in the repository).

crubier commented 4 years ago

I also wonder about the runtime performance impact of accessing JS files inside zip files instead of plain direct access to them.

In the issue https://github.com/yarnpkg/berry/issues/1817 I talked about the performance impact of pnp (4x slower on some use cases), and maybe unzipping files might be a part of this impact? Or maybe not, I have no idea. @arcanis did you measure this at some point?

larixer commented 4 years ago

@crubier

In the issue #1817 I talked about the performance impact of pnp (4x slower on some use cases)

In which cases? I thought we have discussed all the cases presented in that issue and closed it. One case was fsevents - which got fixed. Another case is Webpack build caching that were hardcoded to node_modules approach - more flexible build caching should be used instead. As for "general" PnP runtime slowness, you haven't presented any demos. Please clarify where do you see 4x slowdown?

arcanis commented 4 years ago

I don't have numbers right at hand (last time I checked was about a year ago) but from what I remember zip overhead wasn't that high. While it should be taken with a pinch of salt, in theory it can even lead to improved startup times due to file collocation (same thing as what is described in the SQLite documentation: 35% Faster Than The Filesystem).