search

pastelsky / bundlephobia

🏋️ Find out the cost of adding a new frontend dependency to your project
https://bundlephobia.com
MIT License
9.04k stars 224 forks source link

What’s next for bundlephobia #179

Open pastelsky opened 5 years ago

pastelsky commented 5 years ago

Javascript bundles objects tend to accumulate bloat rather quickly – and a majority of this is often accounted for by 3rd party npm pacakges. Bundlephobia aims to empower devs to make an informed decision about adding such a new dependency.

Who uses bundlephobia?

image

Bundlephobia receives more than 300k requests a day. On the website itself, itself, it receives more than 12k search requests a day. Apart from a few REST API requests it receives, it also powers a badge service, a few browsers and editor extensions.

What’s awesome is that library authors are now using bundlephobia to track sizes of their own packages (and tweeting about it!). I’ve also observed a few libraries that have received issues / PRs that offer suggestions on how the author can reduce sizes with the help from the “Compositions” panel.

What would we like to improve on?

Currently, bundlephobia is hosted on a single Digital Ocean droplet which costs around $960 / year. Digital Ocean has been gracious enough to sponsor this cost for the second year, but we’re going way above their OSS plan limits, and I’d like this service to be self-sustainable.

The Chrome Performance Fund might be able to help with the following –

  1. Hosting costs for Bundlephobia, and a monitoring service for better SLAs
  2. Bounties for new features / bug fixes by contributors
addyosmani commented 5 years ago

Exciting!

Improve alternate package suggestions. Define better criteria, collect and rank community suggestions

I'm very interested in a set of criteria for recommending alternatives (e.g based on size, quality etc) when it makes sense. Did you have any thoughts on how to tackle this? e.g would it be manually curated recommendations (e.g Moment -> date-fns, luxon)? auto?

Alerts for library authors

Curious how this would work. Would it be alerting about usage? Dep sizes going up? I imagine authors may already have some telemetry for this in their builds but I could also be wayy overguessing here :)

pastelsky commented 5 years ago

👋 @addyosmani. We accept recommendations today either suo moto or from the community – so in a sense it is manual and curated. I recently defined the criteria for such submissions through our issue template, which can be refined –

Mind dump

We don't want the recommendations to get too generic. We influence this today by carving out specific categories – for eg. React based slider libraries and Vanilla JS sliders would be two separate categories – since both target different group of developers.

What I did notice however is, that new libraries keep coming up very often, and a category like date-time manipulation or charting would get flooded with good suggestions. A simple 👍/👎 rating system should take care of that. We only want to show the top 4-5 alternatives in any category.

It would be good to have short community contributed (1-para) testimonials/guide about how the alternative fares. Which features it doesn't cover, or if the replacement is a bundle of multiple packages etc. This at least makes sense for popular categories but I'm still debating if it's worth the hassle of maintenance.

Once we have a good database of suggestions, it would be easy to build lint plugins to offer these suggestions to users closer to home.

Here's how the current effort looks like.

pastelsky commented 5 years ago

About alerting authors – this is better solved with bundlesize if the author wishes to integrate it. But I was thinking of (and again, just a though dump) a lower commitment solution wherein a package author / interested party can enter the name of an npm package, and subscribe to size notification whenever a new version is published (2 clicks). Maybe even twitter shoutouts.

addyosmani commented 5 years ago

Thanks for sharing all of this detail, @pastelsky! Super helpful :)

We accept recommendations today either suo moto or from the community – so in a sense it is manual and curated. I recently defined the criteria for such submissions through our issue template, which can be refine

This seems like a nice balance. I'm curious if you could share how the current weighting works. Is this perhaps based on how many of the above criteria are met?

A simple / rating system should take care of that. We only want to show the top 4-5 alternatives in any category.

I like the idea of a rating system for highlighting the top 5 alternatives. It appears there are already a few categories where there are up to 7 alternatives already so this seems like it would be a well-timed exploration :)

Some additional thoughts:

Help folks select an alternative

If we take an example of a popular library like Moment.js, users are presented with 3-4 alternatives when visiting the package page. Short of differences in size, it's really difficult to tell how these alternatives are different. e.g how do luxon and date-fns differ if I'm currently using Moment?.

image

Perhaps there are ways we could highlight specific similarities and/or differences? One option would be encouraging other submissions for alternatives include a few more details. e.g timezones, locales:

Name Size(gzip) Tree-shaking Popularity(stars) Methods richness Pattern Timezone Support Locale
Moment.js 329K(69.6K) No 39k High OO Good(moment-timezone) 123
Luxon 59.9K(17.2K) No 7k High OO Good(Intl) -
date-fns 78.4k(13.4k) without tree-shaking Yes 15k High Functional Not yet 50
dayjs 6.5k(2.6k) without plugins No 20k High OO Not yet 130

This is based on data from: https://github.com/you-dont-need/You-Dont-Need-Momentjs#brief-comparison

Linting plugin (x-ref https://github.com/pastelsky/bundlephobia/issues/192)

Once we have a good database of suggestions, it would be easy to build lint plugins to offer these suggestions to users closer to home.

It would be really powerful if there was a BundlePhobia webpack, babel or ESLint plugin that could lint for usage of popular large dependencies and suggest alternatives. Something that could potentially integrate with an npm/yarn preinstall step so maybe even recommendations are made before users perform an install.

Something like:

$ npm install moment 
$ > bundle-lint found 3 smaller alternatives for your consideration. 
> How would you like to proceed?
[  ] install luxon [19.29KB min/gzip - 71% smaller]
[  ] install date-fns [2.74KB min/gzip - 90% smaller]
[  ] install dayjs [6.54KB - 90% smaller]
[  ] install moment

(could even include brief notes on which support timezones/locales)

There are also lots of directions one could take linting - you could scan their source/package.json to understand if there are specific features in moment being used and suggest the best "fit", saving the author from more research before making a decision.

$ npm install moment 
$ > bundle-lint found 3 smaller alternatives for your consideration. 
> How would you like to proceed?
[  ] install luxon [19.29KB min/gzip - 71% smaller] - recommended (closest match for you)
[  ] list more alternatives
[  ] install moment

I was thinking of (and again, just a though dump) a lower commitment solution wherein a package author / interested party can enter the name of an npm package, and subscribe to size notification whenever a new version is published (2 clicks). Maybe even twitter shoutouts.

This does sound useful. Perhaps also helpful for package users that want to be notified if one of their frequently used dependencies is getting much larger than it initially was?

pastelsky commented 5 years ago

Recommendations

I'm curious if you could share how the current weighting works. Is this perhaps based on how many of the above criteria are met?

The weighting that you’ve talking about is used for something entirely different. Since we only have 4-5 recommendations in a category at a time, the weighted keywords help us fuzzy-categorize other (unknown) packages, so these recommendations can be shown when they're searched for as well.

It appears there are already a few categories where there are up to 7 alternatives already so this seems like it would be a well-timed exploration :)

Again, the tag list you're seeing there aren't really recommendations. The 'Natural language date-time utilities category, for example, has three recommendations – ['chrono-node', 'its-a-date', 'parse-messy-time']

Disambiguation

If we take an example of a popular library like Moment.js, users are presented with 3-4 alternatives when visiting the package page. Short of differences in size, it's really difficult to tell how these alternatives are different. e.g how do luxon and date-fns differ if I'm currently using Moment?.

That's a very neat table that you've linked here. It is how we want to do it if we do it. The challenge, however, is that libraries change very fast and it's hard to solve for accuracy of content when you start scaling this up. A balanced approach might work here — doing this only for stable and relatively more popular categories.

Hooks

Linting plugin

I really like the idea of a pre-install hook. I think that's the right opportunity to warn developers.

This does sound useful. Perhaps also helpful for package users that want to be notified if one of their frequently used dependencies is getting much larger than it initially was?

How about a pre/post install hook for this? Every time you yarn upgrade/install package(s), we can give you a diff of the package sizes that existed before and after. Say a user had foo@1.2.0 and he does a yarn install foo@1.3.0, the output looks like (a prettier version) of this –

Building fresh packages...
success Saved lockfile.
success Saved a new dependency.
info Direct dependencies

-----------------------------------------
Size changes    Before             After
------------------------------------------
foo             12kB               19kB
-----------------------------------------

⚠️ foo increased by 7kB (minzipped) after the update.

A naive way to do is just to compare the size differences in package versions directly from bundlephobia. Surely, installations are more complex than that and transitive dependencies have their own effects – an upgrade can even decrease your bundle size because the transitive dependencies align better with rest of the packages you use.

Hence a more accurate way would be to diff yarn.lock and give a size report about all of the changed deps.

So to summarize, I think here's what I'm excited about exploring from our discussions till now –

addyosmani commented 5 years ago

That's a very neat table that you've linked here. It is how we want to do it if we do it. The challenge, however, is that libraries change very fast and it's hard to solve for accuracy of content when you start scaling this up. A balanced approach might work here — doing this only for stable and relatively more popular categories.

I fully agree on the scaling challenges of this approach and like the idea of only doing it for more stable/popular categories.

How about a pre/post install hook for this? Every time you yarn upgrade/install package(s), we can give you a diff of the package sizes that existed before and after. Say a user had foo@1.2.0 and he does a yarn install foo@1.3.0, the output looks like (a prettier version) of this

Oh, I do like the idea of a diff during pre/post install. I could see this being valuable for a lot of developers considering upgrades to their dependencies.

Comparisons tables in popular curated categories to make choosing easier.

👍

A plugin that integrates with the install lifecycle and makes the following possible - Proactive recommendations (blocking?) when you try to install a dependency that has better alternate options available. Gives size change report of packages (direct or transitive) when adding/removing/ updating packages.

+1. Imo there would be high value in better recommendation tooling during install and I could see size reports offering value to many users.

pastelsky commented 5 years ago

Sounds good @addyosmani.

addyosmani commented 5 years ago

The Chrome Performance Fund might be able to help with the following

Hey @pastelsky. I'm happy to share Chrome is donating $10,000 to support bundlephobia's roadmap 🎉 We think project's goals of helping find deps that support tree-shaking or just have a smaller size footprint can have a big impact on JS bundle sizes on the web.

In no particular order, we're excited to see the project explore the proposed work on:

  1. Improving alternate package suggestions (better criteria, rankings, comparison tables)
  2. Hosting costs for bundlephobia, and a monitoring service for better SLAs
  3. Proactive recommendations (perhaps a bundlephobia linting plugin for recommending alternatives to large deps during the npm/yarn preinstall phase)
  4. Creating a first class REST API to access bundlephobia's knowledge base

with funding helping with hosting and bounties to help cover some of this work.

Amazing work on the project, @pastelsky and team. We regularly use bundlephobia and hope this helps just a little :)

~ signed Addy, @stubbornella and @spanicker

pastelsky commented 5 years ago

🎊🎊🎊 This is amazing news for the future of bundlephobia and anyone looking to become a part of this journey – contributing through ideas, design or code.

I hope to more clearly define the proposed work items in more detail in the coming days. Cheers!

avindra commented 5 years ago

Comparisons tables in popular curated categories to make choosing easier.

This would be incredible for discovery. I would like to elaborate on a simpler ask:

A feature to just manually compare two packages. Installing new packages is fine, but what about when we replace packages? Then you are probably interested in the difference in size.

Being able to see side by side comparisons for packages would give us the ability to share one URL with teammates to share the difference (accelerating the use case here-- understanding the delta in bundle sizes).

muuvmuuv commented 5 years ago

Things that came in my mind using bundlephobia; improvements, recommendations and new features:

Quite a lot... sorry! Don't take everything tooo seriuos. I have just listed things that I would add and to provide some stuff to think about. Also, I think it's good to have a list of everything ^^.

Anyway, thank you! Really good idea and something that could help a lot in future and existing projects to optimize or micro-optimize projects. :)

pastelsky commented 5 years ago

Thanks for the suggestions @muuvmuuv . I don't intend on eeking out more performance gains just yet – I think bundlephobia is pretty well performant (with room for more) and time spent doing so would meantime not spent solving some other problems (bundlephobia has quite a few). Also, I currently do not intend bundlephobia to be the place to discover new packages (stars / downloads / popularity etc) since npmjs.com and yarnpkg.com solve this quite well today.

But some of them might make its way to Bundlephobia soon – API tokens / about page etc

pastelsky commented 5 years ago

For anyone wanting to keep track of features as we release new ones – we now publish a reader-friendly summary of changes here – https://github.com/pastelsky/bundlephobia/releases

dy commented 4 years ago

May I propose considering authors' reputation for alternative packages, some reputable/known names are more trusted than the others. I was collecting good analogs for some time already https://github.com/dy/analogs, not sure if that is of any interest.

amark commented 4 years ago

@pastelsky @addyosmani so how does this work? How do projects get included?

I run a popular 12K+ starred MIT/ZLIB/Apache2 alternative to Firebase that is self-hosted, fast, and used in production on some large sites/non-profits.

It would mean a lot to be included, I don't ask developers or companies to pay anything, and begging for sponsorships/VC just distracts time away from improving performance/scaling and features which is what I try to focus on. Or if you have recommendations on some podcasts you could help get me on, that'd also be <3.

Thanks for bundlephobia and excited to see it integrated into Chrome devtools!

pketh commented 3 weeks ago

as an end-user dev who uses bundlephobia to evaluate libs to use (like a premium/good npm), I would pay for ad hoc access (e.g. a package of n lookups) or a subscription ( n lookups/mth if it's low) to be able to reliably use bundlephobia.

It's one of those tools I don't need to often, but when I need it, I really feel it's absence