search

samvera-labs / active_encode

Declare encode job classes that can be run by a variety of encoding services
Other
6 stars 8 forks source link

Size of distributed rubygem is ballooning, presumably accidentally #126

Closed jrochkind closed 1 year ago

jrochkind commented 1 year ago

The size of the distributed rubygem package sent to rubygems.org has been ballooning. Copied from https://rubygems.org/gems/active_encode:

1.2.0 - May 10, 2023 (170 MB) 1.1.3 - March 28, 2023 (74.8 MB) 1.1.2 - January 30, 2023 (74.8 MB) 1.1.1 - December 05, 2022 (16.5 MB)

From 16MB to 170MB in five months! 16MB is already very large for a gem, many are under 1MB. Ruby source code isn't usually very large.

I noticed this when I upgraded from 1.1.1 to 1.2.0, with the extra 155MB of weight... and it was big enough that heroku, my PaaS where I deploy, started warning me that my build/release size was awfully big.

Did some debugging to try to figure out why, identified active-encode.

My guess is that some test files or something like that are being included in the distributed rubygem when they don't have to be?

Tomorrow I will try to continue debugging and figure out why (it might be obvious once I start looking at it), but if anyone who sees this has an idea to share before I get there, please feel free!

jrochkind commented 1 year ago

Yep, okay, it's presumably the it's the ~172M of content in spec/fixtures.

That does not need to/should not be included in the distributed rubygem -- we don't want to make every application using active-encode carry with it 172M of fixture data on deploy!

jrochkind commented 1 year ago

Also though... could we test with smaller fixture files? Do we REALLY need 172M of fixtures in the repo?

Even with smaller fixture files, we probably still want to exclude them from rubygems distribution, to make the total size nice and slim. But even with excluding from rubygems distro.... do we really need all that in the repo?

It looks like possibly some of those fixture files were added by @masaball in recent commits. @masaball, do you think there's any way to replace those fixture files with smaller files that can still demonstrate the same thing, figure out how to make the files as small as possible to fulfill their test purpose?