vanessayuenn commented 2 months ago

🧑‍🤝‍🧑 Who: @JReinhold and @ndelangen

This is a tracking issue for the Reduce install footprint 🐾 project. The purpose of this issue is to keep tracking of the overall status of the project and tasks, and plan everything around it.

See also #29072 which is a spike we'll be doing as part of this project to investigate how to move Storybook towards publishing ESM-only packages.

📢 Want to help?

Make sure you read this issue thoroughly to understand what we want to achieve and how. Do not ask to be assigned to certain tasks, just do them. We won't "reserve" anything to potential contributors, that system rarely work. Help out with what you want, and report findings in this issue, eg. if you've identified potential optimisations in a package. If you open pull requests with your work make sure to reference this issue and tag @JReinhold.

⚠️ Problem

One of the biggest complaints about Storybook is that it's a big and heavy dependency to add to your project. There are muliple ways to interpret the frustration, but one of the most impactful improvements we can make is to reduce the install size of the core Storybook experience.

Throughout the current Storybook 8 releases we've already managed to cut the size significantly, eg. via #27039 and #28519. But there is plenty more work to be done - especially outside the core package - that will enable us to shrink Storybook even further.

🏁 Goals

The high-level goals of this project is to:

Reduce the install size of "the core Storybook experience". That is, the packages that are included in a normal init process, such as storybook, the builders (Vite/Webpack), the renderers (React, Svelte, etc.) and the default addons.
... this includes reducing the dependency graph of mentioned packages, to speedup installation.
Set long-term baselines and goals that will keep us in check, ensuring we don't regress the install-size later.
Build and set up tooling to support these goals. We'll try to find ways that make it easier to maintain lean packages and reduce their footprint. This includes easy-to-access bundle and package analysis tools and reporters.
Identify changes that require a major version bump. This project will conclude before the next major version of Storybook, so we can't make those changes now, but we can prepare them so they are ready when we're closer to the next major release.

This means we are not focusing on performance improvements nor reducing the size of a built Storybook. We still love those improvements, but they are not the major focus here.

📚 Resources

e18e.dev - provides great guides on how to cleanup, speedup and levelup (?😅) packages.
https://github.com/styfle/packagephobia#how-is-this-different great list of different tools in this space that all achieve different things
https://github.com/TheDevMinerTV/package-size-calculator
https://github.com/tinylibs/
https://github.com/unjs/
https://github.com/es-tooling/module-replacements/blob/main/docs/modules/README.md

📊 Spreadsheet with package stats

🔬 Methodology

Storybook is a complex mesh of over 30 packages, therefore analysing "Storybook's" install size will be tackled from two angles:

E2E tests aka sandboxes: Storybook sandboxes represent real-world, minimal projects with Storybook installed. Storybook support countless project configurations, eg. A NextJS project, or a Vite-based Vue project. By initialising Storybook a top these minimal projects, we get a sense of what impact Storybook has on the overall node_modules size and the amount of dependencies added to the project. The upside of this perspective is that it directly focuses on the experience our users get. The downside of this approach - as with most E2E tests - is that it can be "flaky" because a lot of outside factors can impact the project's install size and dependency count. It's also a coarse metric, detecting significant size increases in sandboxes doesn't really help us identify where the increase is coming from. This brings us to...
Unit tests aka packages: Each individual Storybook package will have its own metric on install size and dependency graph. Measuring and detecting size increases at the package-level makes it a lot easier to pinpoint the cause of it. But laser-focusing on packages in isolations can lead to work that makes little difference to the end-user, because they don't use the Storybook packages in isolation. Eg. removing a big dependency from @storybook/react-vite doesn't help if that dependency is still installed by @storybook/core.

Therefore we must keep both metrics in mind, to ensure we improve the experience for the end-user while still getting actionable results on the lower level.

We'll use a breadth-first approach when doing the optimisations across the packages. At the start of the project all, the integration packages have a single task (which also applies to the core): Identify potential optimisations. We'll do that first across all the packages on a high-level, to identify common patterns and find big wins first. This ensures that we don't laser focus on eg. removing 15 KB from @storybook/react when we could instead have removed 12 MB from @storybook/vue3-vite. With that said, there are already known big wins in the core packages that might get a higher priority than looking into some of the integration packages.

There are many approaches and tools to optimising package's size and dependencies and it's not a one-size-fits-all. https://e18e.dev/guide/cleanup.html is a good primer on how to approach it. At a high-level we'll:

Use https://pkg-size.dev to identify and monitor the size of the packages and their dependencies
Use https://npmgraph.js.org to identify and monitor the packages' dependency graph
Use ESBuild's metafiles that are produced when building all the packages, to identify and monitor the packages' internal structure and bundled dependencies.

We can take @storybook/core as an example:

https://pkg-size.dev/@storybook/core@8.3.0-beta.4 shows that there are two major esbuild dependencies significantly contributing to the size. Can we replace it? See #29082
https://npmgraph.js.org/?q=@storybook/core@8.3.0-beta.4#select=exact%3Aexpress%404.20.0 shows that express (among others) contribute to a big part of the dependency graph. Can we replace it? See #29083
ESBuild's metafiles for @storybook/core (not currently public) shows that Prettier makes up a big part of the bundled output. Can we replace it, or remove it? See #29084

🚩 Milestones

📈 Baselines and Bechmarks

See spreadsheet detailing all the packages.

### Tasks
- [ ] https://github.com/storybookjs/storybook/issues/29099
- [ ] https://github.com/storybookjs/storybook/issues/29100
- [ ] qweqwe

🔧 Optimisations

This section includes all the actual package optimisations that we want to make. The list is highly dynamic and should change a lot during the project.

### Architectural changes
- [ ] https://github.com/storybookjs/storybook/issues/29084
- [ ] https://github.com/storybookjs/storybook/issues/29166
- [ ] https://github.com/storybookjs/storybook/issues/29164
- [ ] ~Re-establish CJS auto-deduplication with esbuild using [cjs-splitting.ts from tsup](https://github.com/egoist/tsup/blob/796fc5030f68f929fecde7c94732e9a586ba7508/src/plugins/cjs-splitting.ts)~
- [ ] https://github.com/storybookjs/storybook/issues/29217

📦 Core Packages

```[tasklist] ### `@storybook/core` Optimizations - [x] Identify potential optimisations - [ ] https://github.com/storybookjs/storybook/issues/29082 - [ ] https://github.com/storybookjs/storybook/issues/29083 - [ ] #28262 - [ ] #28315 - [ ] #28611 - [ ] #28981 - [ ] #28663 - [ ] https://github.com/storybookjs/storybook/issues/29167 - [ ] https://github.com/storybookjs/storybook/issues/29252 - [ ] https://github.com/storybookjs/storybook/issues/29168 - [ ] https://github.com/storybookjs/storybook/issues/29104 - [ ] https://github.com/storybookjs/storybook/pull/29126 - [ ] https://github.com/storybookjs/storybook/issues/29227 - [ ] https://github.com/storybookjs/storybook/issues/29103 - [ ] https://github.com/storybookjs/storybook/issues/29229 - [ ] https://github.com/storybookjs/storybook/issues/29161 - [ ] Investigate migrating away from `@yarn/ziplib` and friends - [ ] https://github.com/storybookjs/storybook/issues/29143 ``` ```[tasklist] ### `create-storybook` Optimizations - [x] Identify potential optimisations - [ ] Don't depend on `storybook` in `create-storybook` - [ ] Don't depend on `prettier` to autoformat the main-config - [ ] Prebundle dependencies in `create-storybook` - [ ] Document `npm create storybook` instead of `npx storybook init` in docs ``` 👆 Doing all of the above `create-storybook` tasks should result in reducing its install size **from 73 MB to <1.8 MB** and the dependency count **from 172 to ~3**, greatly reducing the time-to-init. A counter argument is that a lot of these dependencies comes from `storybook` which could be globally cached by the package manager. So taking `storybook` out of `create-storybook` would just move the download of that package to from pre-init to post-init - but it's just a theory. ```[tasklist] ### `@storybook/source-loader` Optimizations - [ ] Identify potential optimisations ```

🧩 Integration Packages

Builders

```[tasklist] ### `@storybook/builder-vite` - [x] Identify potential optimisations - [x] Prebundle dependencies ``` ```[tasklist] ### `@storybook/builder-webpack5` - [x] Identify potential optimisations - [ ] https://github.com/storybookjs/storybook/issues/29181 - [ ] Prebundle dependencies - [ ] Discuss making Webpack a peer dependency of `builder-webpack5` in 9.0, similar to `builder-vite` - [ ] Discuss dropping support for checking types with `fork-ts-checker-webpack-plugin` - [ ] Discuss merging all the preset packages into the frameworks (eg. merge `@storybook/preset-react-webpack` into `@storybook/react-webpack5`) - [ ] Discuss merging `@storybook/core-webpack` into `@storybook/builder-webpack5` ```

Renderers

```[tasklist] ### `@storybook/react` - [x] Identify potential optimisations - [x] Cleanup dependencies - remove unused and move `@types` to `devDependencies` - [x] Prebundle dependencies ``` ```[tasklist] ### `@storybook/vue3` - [x] Identify potential optimisations - [ ] Cleanup dependencies - remove unused and move `@types` to `devDependencies` - [ ] Prebundle dependencies ``` ```[tasklist] ### `@storybook/web-components` - [x] Identify potential optimisations ```

Frameworks

```[tasklist] ### `@storybook/react-vite` - [x] Identify potential optimisations ``` ```[tasklist] ### `@storybook/react-webpack5` - [x] Identify potential optimisations ``` ```[tasklist] ### `@storybook/nextjs` - [x] Identify potential optimisations - [ ] https://github.com/storybookjs/storybook/issues/29185 - [ ] https://github.com/storybookjs/storybook/issues/29195 - [ ] https://github.com/storybookjs/storybook/issues/29188 - [ ] Prebundle possible dependencies in `@storybook/nextjs` ``` ```[tasklist] ### `@storybook/experimental-nextjs-vite` - [x] Identify potential optimisations ``` ```[tasklist] ### `@storybook/angular` - [ ] Identify potential optimisations ``` ```[tasklist] ### `@storybook/vue3-vite` - [ ] Identify potential optimisations ``` ```[tasklist] ### `@storybook/vue3-webpack5` - [ ] Identify potential optimisations ``` ```[tasklist] ### `@storybook/sveltekit` - [ ] Identify potential optimisations ``` ```[tasklist] ### `@storybook/svelte-vite` - [ ] Identify potential optimisations ``` ```[tasklist] ### `@storybook/web-components-vite` - [ ] Identify potential optimisations ``` ```[tasklist] ### `@storybook/web-components-webpack5` - [ ] Identify potential optimisations ```

Addons

```[tasklist] ### `@storybook/addon-docs` - [x] Identify potential optimisations ``` ```[tasklist] ### `@storybook/blocks` - [x] Identify potential optimisations ``` ```[tasklist] ### `@storybook/addon-interactions` - [x] Identify potential optimisations ``` ```[tasklist] ### `@storybook/addon-actions` - [x] Identify potential optimisations ``` ```[tasklist] ### `@storybook/test` - [x] Identify potential optimisations ``` ```[tasklist] ### `@storybook/addon-controls` - [ ] Identify potential optimisations ``` ```[tasklist] ### `@storybook/addon-measure` - [ ] Identify potential optimisations ``` ```[tasklist] ### `@storybook/addon-outline` - [ ] Identify potential optimisations ``` ```[tasklist] ### `@storybook/addon-backgrounds` - [ ] Identify potential optimisations ``` ```[tasklist] ### `@storybook/addon-toolbars` - [ ] Identify potential optimisations ``` ```[tasklist] ### `@storybook/addon-viewport` - [ ] Identify potential optimisations ``` ```[tasklist] ### `@storybook/addon-onboarding` - [ ] Identify potential optimisations ```

🛠️ Supporting Tooling

Ideally we'd want to set up some tooling that allows to easily inspect package and dependency sizes, to make it easier to optimise these areas. It's still TBD what shape or form these could take.

### 🌟 Essential tooling
- [ ] https://github.com/storybookjs/storybook/issues/29105
- [ ] Improve readability and usefulness of existing sandbox benchmark reports on PRs
- [x] Reduce the friction of inspecting packages' build outputs
- [ ] https://github.com/storybookjs/storybook/issues/29322

### ✨ Nice-to-have tooling
- [ ] https://github.com/storybookjs/storybook/issues/29251
- [ ] Reduce the friction of inspecting packages' dependency graph
- [ ] https://github.com/storybookjs/storybook/issues/29106

🤷 Misc

### 🎁 Wrap up
- [ ] Create a new tracking issue containing the known optimisations we didn't had time to do in this project
- [ ] Compile a list of optimisations we want to do for 9.0 👇
- [ ] Publish a blog post about the reduced footprint

### 💥 Breaking optimisations
- [ ] https://github.com/storybookjs/storybook/issues/29159
- [ ] https://github.com/storybookjs/storybook/issues/29160

### 💖 Nice-to-haves
- [ ] Investigate the impact on running `storybook dev` at the end of `storybook init`
- [ ] https://github.com/storybookjs/storybook/issues/29073

JReinhold commented 2 months ago

Based on the baseline calculations, I think it's fair to skip the following packages along with @storybook/addon-essentials for now, because they are "tiny" compared to the rest of the packages here:

TheThing commented 2 months ago

Just a small heads up, I think a rogue file might have snuck its way into the release tarballs ;) I downloaded the latest tarball release (8.3.2) straight from npm and there seems to be a rogue __mocks__ that snuck through. Not a big deal but I figured I should report it :)

mynd

pumano commented 1 month ago

Maybe https://github.com/ai/size-limit helps with process to automate code + deps size on pull requests

ravicious commented 1 month ago

If I can chime in, #25387 already moved @types/node from a dep to a dev dep in core-common, but I see that @storybook/react still includes @types/node as a dep. https://github.com/storybookjs/storybook/issues/24873

See also https://github.com/SBoudrias/Inquirer.js/pull/1569. This was the dep that caused msw to include @types/node as a dep. They've recently fixed that issue and msw was updated not to do that. When upgrading msw, I realized that @storybook/react is another package in our package.json that does this.

JReinhold commented 1 month ago

@ravicious I believe that has already been fixed?

https://github.com/storybookjs/storybook/blob/next/code%2Frenderers%2Freact%2Fpackage.json