search

CircleCI-Public / cimg-ruby

The CircleCI Ruby Docker Convenience Image.
https://circleci.com/developer/images/image/cimg/ruby
MIT License
33 stars 39 forks source link

Delete jemalloc flag in template dockerfile #156

Closed ksylvest closed 10 months ago

ksylvest commented 10 months ago

For our official CircleCI Docker Convenience Image support policy, please see CircleCI docs.

This policy outlines the release, update, and deprecation policy for CircleCI Docker Convenience Images.

Description

This flag is not present in any other Dockerfiles and prevents compiling gems using native extensions (e.g. bcrypt, pg, etc). It looks like it was introduced in https://github.com/CircleCI-Public/cimg-ruby/pull/139 (possibly without an image being built). Opting to delete just flag, but a followup PR is probably needed to cleanup or fix the remainder. As it stands the 3.3 images aren't able to install gems with native extensions (e.g. pg, nokogiri, bcrypt, etc).

Before

gem install bcrypt
Fetching bcrypt-3.1.20.gem
Building native extensions. This could take a while...
ERROR:  Error installing bcrypt:
    ERROR: Failed to build gem native extension.

    current directory: /home/circleci/.rubygems/gems/bcrypt-3.1.20/ext/mri
/usr/local/bin/ruby extconf.rb
creating Makefile

current directory: /home/circleci/.rubygems/gems/bcrypt-3.1.20/ext/mri
make DESTDIR\= sitearchdir\=./.gem.20240102-13-aa2evc sitelibdir\=./.gem.20240102-13-aa2evc clean

current directory: /home/circleci/.rubygems/gems/bcrypt-3.1.20/ext/mri
make DESTDIR\= sitearchdir\=./.gem.20240102-13-aa2evc sitelibdir\=./.gem.20240102-13-aa2evc
compiling bcrypt_ext.c
In file included from /usr/local/include/ruby-3.3.0/ruby/internal/config.h:22,
                 from /usr/local/include/ruby-3.3.0/ruby/ruby.h:15,
                 from /usr/local/include/ruby-3.3.0/ruby.h:38,
                 from bcrypt_ext.c:1:
/usr/local/include/ruby-3.3.0/x86_64-linux/ruby/config.h:82:40: fatal error: jemalloc/jemalloc.h: No such file or directory
   82 | #define RUBY_ALTERNATIVE_MALLOC_HEADER <jemalloc/jemalloc.h>
      |                                        ^
compilation terminated.
make: *** [Makefile:248: bcrypt_ext.o] Error 1

make failed, exit code 2

Gem files will remain installed in /home/circleci/.rubygems/gems/bcrypt-3.1.20 for inspection.
Results logged to /home/circleci/.rubygems/extensions/x86_64-linux/3.3.0/bcrypt-3.1.20/gem_make.out

After

gem install bcrypt
Fetching bcrypt-3.1.20.gem
Building native extensions. This could take a while...
Successfully installed bcrypt-3.1.20
1 gem installed

Reasons

https://github.com/CircleCI-Public/cimg-ruby/issues/155

Checklist

Please check through the following before opening your PR. Thank you!

ryanbourdais commented 10 months ago

Hi @ksylvest can you remove the changes in 3.3/Dockerfile? We only make manual edits in the template. This all looks fine to me to merge in, but these changes won't be reflected in the 3.3 release as we don't re-release images, unless we receive further reports of the image being unusable or breaking a large number of builds.

ryanbourdais commented 10 months ago

looking into this more, does this issue arise when you install libjemalloc beforehand? It seems to me that this issue can actually be solved by removing a reference to libjemalloc here

ksylvest commented 10 months ago

@ryanbourdais I'd suggest you consider re-publishing. The existing image is fairly unable in the current state:

  1. The rails gem (used for installing Ruby on Rails) fails to install due to dependencies on websocket-extensions with >480 million downloads.
  2. The pg gem is the default Postgres gem with >280 million downloads.

  3. The bcrypt gem is a ubiquitous password hashing algorithm with >200 million downloads.

    Per your inquiry, I'd also suggest yanking the change instead of attempting to fix with the broken image as it stands. I think that sort of change might merit some QA ahead of merging to avoid customers needing to debug broken images. Maybe a subsequent release and for now just go back to a knowing working configuration?

ryanbourdais commented 10 months ago

@ksylvest do you have build links to those gem failing to install? Also, can you remove the changes to 3.3/Dockerfile, we dont make those changes manually in PRs such as this, we will re-generate the dockerfiles from the template if we re-release this image. Can you also remove libjemalloc-dev from the statement on live 94 sudo apt-get remove rustc libstd-rust* libjemalloc-dev so that customers can still enable jemalloc themselves if they need it? I believe that you're right and we will likely have to re-release this image.

ksylvest commented 10 months ago

@ryanbourdais sure - swapped. Assuming you'll update the Dockerfile when re-publishing? Do you want that split into a PR similar to https://github.com/CircleCI-Public/cimg-ruby/pull/137?

ksylvest commented 10 months ago

Flagging issue to include some tests against images build: https://github.com/CircleCI-Public/cimg-ruby/issues/157 - not sure if any other cimg images have tests already that might be copied / pasted as a template.

ryanbourdais commented 10 months ago

@ksylvest thanks for the changes, I'll run our release script and get this rebuilt soon after they are merged in.

ksylvest commented 10 months ago

Thanks - just to confirm I'm not authorized to merge pull requests.

ryanbourdais commented 10 months ago

Thanks - just to confirm I'm not authorized to merge pull requests.

You should be able to after the tests run, I had to trigger the pipeline since you aren't a listed maintainer of this repository