Support request for native dependencies in ruby

blijblijblij commented 3 years ago

When adding a gem that needs extensions to be natively build the ruby:alpine image seems to be a bit to lightweight and can not build the gems defined within the Gemfile:

source 'https://rubygems.org'

gem 'kubeclient'
gem 'logging'

Expected Behaviour

bundle install
Using rake 13.0.3
Using public_suffix 4.0.6
Using addressable 2.7.0
Using bundler 2.1.4
Using unf_ext 0.0.7.7
Using unf 0.1.4
Using domain_name 0.5.20190701
Using ffi 1.14.2
Using ffi-compiler 1.0.1
Using http-cookie 1.0.3
Using http-form_data 2.3.0
Using http-parser 1.2.2
Using http 4.4.1
Using http-accept 1.7.0
Using multi_json 1.15.0
Using jsonpath 1.1.0
Using recursive-open-struct 1.1.3
Using mime-types-data 3.2020.1104
Using mime-types 3.3.1
Using netrc 0.11.0
Using rest-client 2.1.0
Using kubeclient 4.9.1
Using little-plugger 1.1.4
Using logging 2.3.0
Bundle complete! 2 Gemfile dependencies, 24 gems now installed.
Use `bundle info [gemname]` to see where a bundled gem is installed.

Current Behaviour

...
#52 5.030 Gem::Ext::BuildError: ERROR: Failed to build gem native extension.
#52 5.030 
#52 5.030     current directory: /usr/local/bundle/gems/unf_ext-0.0.7.7/ext/unf_ext
#52 5.030 /usr/local/bin/ruby -I /usr/local/lib/ruby/3.0.0 -r
#52 5.030 ./siteconf20210103-1-9zy4ek.rb extconf.rb
#52 5.030 checking for -lstdc++... *** extconf.rb failed ***
#52 5.030 Could not create Makefile due to some reason, probably lack of necessary
#52 5.030 libraries and/or headers.  Check the mkmf.log file for more details.  You may
#52 5.030 need configuration options.
#52 5.030 
#52 5.030 Provided configuration options:
#52 5.030   --with-opt-dir
#52 5.030   --without-opt-dir
#52 5.030   --with-opt-include
#52 5.030   --without-opt-include=${opt-dir}/include
#52 5.030   --with-opt-lib
#52 5.030   --without-opt-lib=${opt-dir}/lib
#52 5.030   --with-make-prog
#52 5.030   --without-make-prog
#52 5.030   --srcdir=.
#52 5.030   --curdir
#52 5.030   --ruby=/usr/local/bin/$(RUBY_BASE_NAME)
#52 5.030   --with-static-libstdc++
#52 5.030   --without-static-libstdc++
#52 5.030   --with-stdc++-dir
#52 5.030   --without-stdc++-dir
#52 5.030   --with-stdc++-include
#52 5.030   --without-stdc++-include=${stdc++-dir}/include
#52 5.030   --with-stdc++-lib
#52 5.030   --without-stdc++-lib=${stdc++-dir}/lib
#52 5.030   --with-stdc++lib
#52 5.030   --without-stdc++lib
#52 5.030 /usr/local/lib/ruby/3.0.0/mkmf.rb:471:in `try_do': The compiler failed to
#52 5.030 generate an executable file. (RuntimeError)
#52 5.030 You have to install development tools first.
...

Possible Solution

Some github issues mention similar issues, which should be fixed by adding

...
RUN apk add --update \
  build-base \
...

to the Dockerfile

Steps to Reproduce (for bugs)

Using this Gemfile https://github.com/blijblijblij/faas-functions/blob/8664386ee2011b3183631631d15f8d5710539960/node-checker/Gemfile
and while building it with github actions: https://github.com/blijblijblij/faas-functions/runs/1640549064?check_suite_focus=true#step:5:399
the build fails only when I start adding gems that need native building

Context

I am trying to get ruby functions to run on a k3s cluster using openfaas, but just before the finishline my code cannot be wrapped into a container, thus my experimentations are on hold because of it.

Your Environment

Docker version docker version (e.g. Docker 17.0.05 ):


sudo docker version
Client: Docker Engine - Community
Version:           20.10.1
API version:       1.41
Go version:        go1.13.15
Git commit:        831ebea
Built:             Tue Dec 15 04:35:04 2020
OS/Arch:           linux/arm64
Context:           default
Experimental:      true

Server: Docker Engine - Community Engine: Version: 20.10.1 API version: 1.41 (minimum version 1.12) Go version: go1.13.15 Git commit: f001486 Built: Tue Dec 15 04:33:03 2020 OS/Arch: linux/arm64 Experimental: false containerd: Version: 1.4.3 GitCommit: 269548fa27e0089a8b8278fc4fc781d7f65a939b runc: Version: 1.0.0-rc92 GitCommit: ff819c7e9184c13b7c2607fe6c30ae19403a7aff docker-init: Version: 0.19.0 GitCommit: de40ad0


* Are you using Docker Swarm or Kubernetes (FaaS-netes)?
`k3s` on `arm64` and some nodes on `amd64`
* Operating System and version (e.g. Linux, Windows, MacOS):
`ubuntu 20.04`
* Link to your project or a code example to reproduce issue:
https://github.com/blijblijblij/faas-functions/tree/develop/node-checker

blijblijblij commented 3 years ago

Would this https://github.com/blijblijblij/templates/commit/0edb240db27ef3d62e5eb625f32bc99a304bd0f9 fix the issue?!

Not sure if I understood the templates correctly @alexellis

alexellis commented 3 years ago

You need to use a template that is based upon Debian instead of Alpine Linux, which is better suited to pure-Ruby only.

Use the ruby-http one https://github.com/openfaas/ruby-http

blijblijblij commented 3 years ago

Hmm, lots off nuances here, thx for pointing me to ruby-http @alexellis , after some github actions workflow changes I got it to work, although kubeclient gem still seems reluctant to install:

https://github.com/blijblijblij/faas-functions/runs/1648922184?check_suite_focus=true

I'll continue my investigation, thx anyway!

alexellis commented 3 years ago

Have you checked the installation requirements for kubeclient?

https://rubygems.org/gems/kubeclient/versions/3.0.0 https://github.com/abonas/kubeclient

And does it install locally with faas-cli build?

alexellis commented 3 years ago

OK so you need to add this to your stack.yml file:

    build_options:
    - dev

https://docs.openfaas.com/reference/yaml/#function-build-options

That built fine for me:

Step 13/23 : RUN bundle install
 ---> Running in e55f6ecb7162
Fetching gem metadata from https://rubygems.org/........
Resolving dependencies...
Fetching rake 13.0.3
Installing rake 13.0.3
Fetching public_suffix 4.0.6
Installing public_suffix 4.0.6
Fetching addressable 2.7.0
Installing addressable 2.7.0
Using bundler 1.17.2
Fetching unf_ext 0.0.7.7
Installing unf_ext 0.0.7.7 with native extensions
Fetching unf 0.1.4
Installing unf 0.1.4
Fetching domain_name 0.5.20190701
Installing domain_name 0.5.20190701
Fetching ffi 1.14.2
Installing ffi 1.14.2 with native extensions
Fetching ffi-compiler 1.0.1
Installing ffi-compiler 1.0.1
Fetching http-cookie 1.0.3
Installing http-cookie 1.0.3
Fetching http-form_data 2.3.0
Installing http-form_data 2.3.0
Fetching http-parser 1.2.2
Installing http-parser 1.2.2 with native extensions
Fetching http 4.4.1
Installing http 4.4.1
Fetching http-accept 1.7.0
Installing http-accept 1.7.0
Fetching multi_json 1.15.0
Installing multi_json 1.15.0
Fetching jsonpath 1.1.0
Installing jsonpath 1.1.0
Fetching recursive-open-struct 1.1.3
Installing recursive-open-struct 1.1.3
Fetching mime-types-data 3.2020.1104
Installing mime-types-data 3.2020.1104
Fetching mime-types 3.3.1
Installing mime-types 3.3.1
Fetching netrc 0.11.0
Installing netrc 0.11.0
Fetching rest-client 2.1.0
Installing rest-client 2.1.0
Fetching kubeclient 4.9.1
Installing kubeclient 4.9.1
Bundle complete! 2 Gemfile dependencies, 22 gems now installed.
Bundled gems are installed into `/usr/local/bundle`
Removing intermediate container e55f6ecb7162

May I suggest that, if you've found my answers and help useful, that you support our project?

https://github.com/sponsors/openfaas

or

https://github.com/sponsors/alexellis

alexellis commented 3 years ago

/set title: Support request for native dependencies in ruby

blijblijblij commented 3 years ago

Struggled with editing the stack.yml

build_options:
    - dev

Did not work for me, but maybe I placed it in the wrong place, got it to work with ADDITIONAL_PACKAGE while building with github actions, which conceptionally use the same variables:

...
        name: Build and Push the OpenFaaS function
        uses: docker/build-push-action@v2
        with:
          context: ./build/homepage/
          file: ./build/homepage/Dockerfile
          platforms: linux/amd64,linux/arm64
          push: true
          build-args: |
            ADDITIONAL_PACKAGE=build-essential
          tags: |
            blijblijblij/homepage:latest-${{ steps.define_env.outputs.github-sha-short }}
            blijblijblij/homepage:latest
...

Now it works, thanks for your kind help @alexellis

blijblijblij commented 3 years ago

May I suggest that, if you've found my answers and help useful, that you support our project?

https://github.com/sponsors/openfaas

or

https://github.com/sponsors/alexellis

Been there, done that, thanks and keep up the good work! 🚀 🎆

alexellis commented 3 years ago

You cannot use faas-cli build with docker/build-push-action@v2 - it's mutually exclusive.

faas-cli build generates a docker build command, and docker/build-push-action@v2 creates its own docker build command, so you've done the right thing for your CI, if you need it to be multi-arch.

For local development, you need to use the stack.yml approach and faas-cli build, or for multi-arch faas-cli publish

Thanks for the support.

openfaas / templates