search

sinonjs / sinon

Test spies, stubs and mocks for JavaScript.
https://sinonjs.org/
Other
9.65k stars 771 forks source link

File sizes #1410

Closed Ginden closed 7 years ago

Ginden commented 7 years ago

Right now du command shows me that sinon/pkg folder takes almost 6MB of data. Short investigation showed me that: 

$ md5 node_modules/sinon/pkg/*
MD5 (node_modules/sinon/pkg/sinon-2.2.0.js) = 6cc24b579d87d31c10ab4af6413d336c
MD5 (node_modules/sinon/pkg/sinon-no-sourcemaps-2.2.0.js) = 843ffb86d002870674021bb0888316b7
MD5 (node_modules/sinon/pkg/sinon-no-sourcemaps.js) = 843ffb86d002870674021bb0888316b7
MD5 (node_modules/sinon/pkg/sinon.js) = 6cc24b579d87d31c10ab4af6413d336c

So it looks like 1 MB files are duplicated for no apparent reason. Moreover, these files render few editors not responsive because of huge tables of numbers included there.

mantoni commented 7 years ago

This is on purpose as per the build script here: https://github.com/sinonjs/sinon/blob/master/build.js

Some people want to grab files with explicit versions, some just want the latest or use semver and grab whatever is in /pkg/sinon.js.

If you really need to open the Sinon.JS bundle in an editor, use one of the -no-sourcemaps versions because the other files contain source maps, which consist of huge tables of numbers.

In general, we don't think that 6MB of data are an issue for a development dependency. When we changed the packaging for Sinon with v2 we looked at the file sizes and everybody was like ¯\(ツ)/¯

TimvdLippe commented 3 years ago

In general, we don't think that 6MB of data are an issue for a development dependency. When we changed the packaging for Sinon with v2 we looked at the file sizes and everybody was like ¯(ツ)/¯

At ChromeDevTools, we are currently looking into our node_modules size: https://github.com/ChromeDevTools/devtools-frontend/tree/master/node_modules. Sinon is currently the number 5 package in terms of filesize. We are required to checkin our node_modules in our repository for stability and security reasons.

Would it be possible to publish a version of sinon with sourcemaps as a separate package for those who want to run sinon in the browser? The sourcemap information in sinon is a non-trivial amount of data that takes a large amount of disk space. The ~4MB of Sinon has a larger impact, as it will be included on every single Chromium CQ (our continuous integration system) build and I can imagine other projects who run npm i as part of their CI setup will pull in the same amount of data. The data has to be sent over the network and will therefore quickly add up with the hundreds of Chromium CQ builds we perform per day.

TimvdLippe commented 3 years ago

If my calculations are correct, the following would be true:

Currently, Sinon gets downloaded about 3.3M times per week Removing the sourcemap from pkg/sinon.js removes ~1.4MB

3.3M * 1.4MB = 4.62 Terabyte network traffic saved per week (this excludes usages like Chromium where we check in our dependencies, so the actual number is likely much higher). That would be about 1.7+ Petabyte per year in reduced network traffic.

mroderick commented 3 years ago

@TimvdLippe thank you for your thoughts. I'm glad that Sinon is providing value for Google in developing Chrome Dev Tools. I've been a fan of the Chrome Dev Tools ever since the first Chrome stable release.

It is certainly possible to publish several packages from the same repository, but it's not without effort and will need to be maintained over the years.

As I see it, we would need to do the following:

At this point in time, the attention I give to the Sinon libraries is mostly on improving documentation. If I'm working on things on my own and for free, I'm more inclined to work on topics that benefit the many rather than the few.

If you'd like to collaborate and contribute to making the package(s) leaner, that would be great. If you could convince Google to contribute to the Sinon Open Collective that would be much appreciated.

TimvdLippe commented 3 years ago

@mroderick Totally understandable 😄 I don't think this is particularly urgent to work on, but I wanted to reply to the "file size doesn't really matter" by providing some data why it does matter. Hopefully others reading it will not take the statement for granted and consider the other effects of sourcemap shipping.

In any case, I think focusing on the majority use case for Sinon is definitely the right call. If at any point you see an opportunity to tackle this problem, that would be great. But it has no real urgency, given the state of node_modules in general.