search

zakird / wkhtmltopdf_binary_gem

Ruby gem containing easily installable access to wkhtmltopdf application
https://rubygems.org/gems/wkhtmltopdf-binary
Apache License 2.0
182 stars 347 forks source link

Gem size 0.12.5 #55

Open vitalinfo opened 4 years ago

vitalinfo commented 4 years ago

new version of gem much bigger than previous one

0.12.5 - November 07, 2019 (238 MB) 0.12.4 - June 11, 2018 (45.1 MB)

0.12.5 totally unreal use it on heroku

fonji commented 4 years ago

Looking at the merge commit, the cause seems to be that we lost the generic builds (like bin/wkhtmltopdf_linux_amd64) for distribution-with-version-specific ones (like bin/wkhtmltopdf_ubuntu_14.04_amd64.gz). There are 6 builds for ubuntu only…

If this specificity is really necessary, I guess that the install process should wget the correct file only, and it should not be part of the gem directly. Otherwise this is just going to get worse with new distributions and versions, and we'll have a >1Go gem. It would also be nice to leave a generic file for unofficially supported distributions. I mean, I'm on ubuntu 19.04, I had to link to the binary for ubuntu 18.04 and it works fine.

Sorry for not proposing a solution as a pull request, I don't know how to do this exactly and can't put this as a high priority. I'll downgrade.

unixmonkey commented 4 years ago

@vitalinfo Did 0.12.4 work for you on Heroku? Which stack? Does the size of 0.12.5 prevent it from being installed on Heroku? Is there a reason you aren't using https://github.com/rposborne/wkhtmltopdf-heroku for a targeted build?

@fonji I'm not excatly sure why they decided to release multiple versions instead of a generic x86 and x64 fat binary as they have until this version, but I'm sure they have their reasons. I like your suggestion to wget the proper file, but worry that might be difficult to handle errors or assume every system has wget. Why is the size difference enough of a reason to downgrade for you?

sallesma commented 4 years ago

Heroku allows to deploy applications whose weight is maximum 500 MB Having one single gem add 200MB prevents us from deploying (which is enough of a reason to downgrade ;) )

vitalinfo commented 4 years ago

@unixmonkey yes, 0.12.4 works fine on Heroku for us our current slug size 300+Mb, but with 0.12.5 it grows more than 500+Mb and basically it pretty obvious why according to these numbers image

vitormd commented 4 years ago

+200MB on a patch version dooms the version unusable

@vitalinfo Did 0.12.4 work for you on Heroku? Which stack? Does the size of 0.12.5 prevent it from being installed on Heroku? Is there a reason you aren't using https://github.com/rposborne/wkhtmltopdf-heroku for a targeted build?

@fonji I'm not excatly sure why they decided to release multiple versions instead of a generic x86 and x64 fat binary as they have until this version, but I'm sure they have their reasons. I like your suggestion to wget the proper file, but worry that might be difficult to handle errors or assume every system has wget. Why is the size difference enough of a reason to downgrade for you?

wkhtmltopdf-heroku won't work on macOS development enviroment and we understand it wouldn't be a good practice to have different gems for development and production enviroment. This is, so far, the only gem we have'd this problem

unixmonkey commented 4 years ago

@vitormd I disagree that it "dooms the version unusable". I know that it excludes some people due to size and system limitations, but many have been waiting a long time for this update and are happy to have it.

I also disagree that you shouldn't configure your setup to use a different gem for development and production environments, as long as they are the same version of wkhtmltopdf, it shouldn't matter. You are already using different binaries because of the differing platforms, even if they are packaged into the same gem.

This gem merely wraps the wkhtmltopdf released packages, which changed from having a few "universal"-style linux binaries in 0.12.4, to having multiple distribution-specific ones in 0.12.5.

This gem preserves the behavior of prior versions by bundling everything together, whether you need it or not, for convenience, and executing the one that works on your system.

I certainly wish it were smaller, but which binaries should we exclude to make it so?

What's stopping you from pulling out the 2 binaries you need and dropping them into your own project?

I think it might be possible to build our own statically-compiled fat binary for the linux systems, but it would be difficult for me to know for sure it works across all distributions.

I'd happily consider at a PR or in-depth explanation of how to improve the situation.

jorgevbo commented 4 years ago

I'm using AWS Elastic Beanstalk. Application use "amzn_2018.03_amd64" SO. When I use 0.12.5 gem version, log said:

Invalid platform, must be running on Ubuntu 14.04/16.04/18.04 CentOS 6/7, Debian 8/9, or intel-based macOS (missing binary: /opt/rubies/ruby-2.3.8/lib/ruby/gems/2.3.0/gems/wkhtmltopdf-binary-0.12.5/bin/wkhtmltopdf_amzn_2018.03_amd64). (RuntimeError)

So, I downgrade to 0.12.4 gem version

confact commented 4 years ago

I get this error on 0.12.5:

expected no Exception, got #<RuntimeError: Failed to execute:
9228        ["/usr/local/bundle/bin/wkhtmltopdf", "--margin-top", "0", "--marg...r/local/bundle/bin/wkhtmltopdf:23:in `load'
9229            from /usr/local/bundle/bin/wkhtmltopdf:23:in `<main>'

I rolled back to 0.12.4 and it works.

axos88 commented 4 years ago

I certainly wish it were smaller, but which binaries should we exclude to make it so?

All of them. The gem should download the single one actually necessary for the system during the installation, and leave the others out. This would also shrink the gem size to a few kb only, although a note would be welcomed in the Readme.md about the real size, possibly an output message at the end of the bundler run.

axos88 commented 4 years ago

Btw, for people having problem with size on heroku: https://github.com/rposborne/wkhtmltopdf-heroku Just went down from 450MB to 180MB by moving to this gem, which installs only the one necessary for heroku.

javierav commented 4 years ago

@unixmonkey @vitalinfo I think that this gem should behave like the webdrivers gem. Currently is unusable for us due to space limitation.

Nitrodist commented 3 years ago

This affects me still :(

pedrofurtado commented 3 years ago

@unixmonkey I think we can close this issue, because we don't have immediate plans to make this kind of change in gem structure for now 🤝

cgriego commented 3 years ago

Anyone using packaged/vendored gems with Bundler also hits the issue that this gem's size now requires the use of Git/GitHub LFS (large file storage).

Qqwy commented 2 years ago

Anno 2022-08 this is still relevant.

AlexanderFisenko commented 2 years ago

Hey! This article helped me with the same problem https://www.eyupatis.com/how-to-decrease-heroku-slug-size-wkhtmltopdf-edition

mkusaka commented 1 year ago

Hi. I always make use of this gem. I thank the maintainer.

Now, I'm also facing this problem and would like to discuss a mitigating solution.

The mitigation I propose is to cut support.

This gem currently support many distributions, but some of which are already EOL.

Now, if we exclude the EOL ones (but keep the latest ones for debian and centos), we can reduce the size by about 110MB

# before
❯ du -h -d 1 | grep bin
263M . /bin

# after
❯ du -h -d 1 | grep bin
159M . /bin

Also, we are aware that i386 usage is relatively low at this moment, and if we exclude them, we can reduce it by another 50 MB or so

❯ du -h -d 1 | grep bin
110M . /bin

I know this is not a perfect solution, but I believe it is a mitigation for many cases!

Is it possible to include this mitigation?

Also, the suggestions so far have been mitigations by cutting support, but we may as well consider mitigations by providing a per-platform gem like the nokogiri gem.

ngan commented 4 months ago

FWIW, you could do...

# Install the gem
bundle
# Execute the binary once so that it decompresses the correct OS-based binary
bundle exec wkhtmltopdf --version
# Remove all the compressed binaries afterwards
rm -rf $(bundle show wkhtmltopdf-binary)/bin/*.gz

You can put the above in your Dockerfile so that your final image is smaller. At the end of the day, I think this gem could leverage arch-based distributions that some other gems use: https://rubygems.org/gems/nokogiri or https://rubygems.org/gems/sqlite3/versions

image

Bundle will install the right gem variant/release based on your system arch. Each one would house just 1 binary.