search

jordansissel / fpm

Effing package management! Build packages for multiple platforms (deb, rpm, etc) with great ease and sanity.
http://fpm.readthedocs.io/en/latest/
Other
11.14k stars 1.07k forks source link

fpm creates the RPMs of gem packages with extensions in their package directories and not in /usr/lib64 #1779

Open sreerajkksd opened 3 years ago

sreerajkksd commented 3 years ago

It looks like the RPMs generated by fpm for ruby gem packages which contain extensions are put directly under /usr/share/gems and not under /usr/lib64.

eg: you can run fpm-s gem -t rpm ffi and that would create an RPM package when installed create the extensions under:

[root@rhel8test04 ~]# rpm -ql  rubygem-ffi | grep ffi_c.so
/usr/share/gems/extensions/x86_64-linux/2.5.0/ffi-1.12.2/ffi_c.so
/usr/share/gems/gems/ffi-1.12.2/ext/ffi_c/ffi_c.so

However, the packages build by redhat or by gem (by default) install these kind of so files under /usr/lib64.

[root@rhel8test04 ~]# sudo rpm -ql rubygem-json-2.1.0-106.module+el8.3.0+7153+c6f6daa5.x86_64 | grep parser.so
/usr/lib64/gems/ruby/json-2.1.0/json/ext/parser.so

Can someone suggest a resolution ? or, Is this a bug in fpm ?

jordansissel commented 1 year ago

@sreerajkksd Hi! I don't necessarily consider this a bug. fpm will ask Ruby for the default install location by running the gem env gemdir command -- https://github.com/jordansissel/fpm/blob/56a97c43be6e9cbdf6c2f5c64920312e623a644f/lib/fpm/package/gem.rb#L226

You can choose your own directory, such as /usr/lib64, by using the --prefix flag in fpm. For example:

% fpm -s gem -t rpm --prefix /usr/lib64 ffi
Created package {:path=>"rubygem-ffi-1.15.5-1.x86_64.rpm"}

% rpm -qlp rubygem-ffi-1.15.5-1.x86_64.rpm | grep ffi_c.so
/usr/lib64/extensions/x86_64-linux/3.1.0/ffi-1.15.5/ffi_c.so
/usr/lib64/gems/ffi-1.15.5/ext/ffi_c/ffi_c.so
/usr/lib64/gems/ffi-1.15.5/lib/ffi_c.so

Let me know if this helps? :)

bugfood commented 1 year ago

I ran into this when building an RPM for the pg gem.

fpm -s gem -t rpm --version 1.4.6 pg

When installed, there is an error:

$ ruby -e "require 'pg'"
Ignoring pg-1.4.6 because its extensions are not built. Try: gem pristine pg --version 1.4.6
Traceback (most recent call last):
    2: from -e:1:in `<main>'
    1: from /usr/share/rubygems/rubygems/core_ext/kernel_require.rb:59:in `require'
/usr/share/rubygems/rubygems/core_ext/kernel_require.rb:59:in `require': cannot load such file -- pg (LoadError)

I don't think this can be fixed with --prefix. To summarize, the problem cannot be solved by installing the entire gem to /usr/lib64. Only the binary extensions (and metadata) get installed there, and not within an extensions dir.

See the contents of the upstream RPM, for example:

$ rpm -ql rubygem-pg
/usr/lib/.build-id
/usr/lib/.build-id/fb
/usr/lib/.build-id/fb/aadc682f9a49406ab4ef6c21cac86cba2858d1
/usr/lib64/gems/ruby/pg-1.0.0
/usr/lib64/gems/ruby/pg-1.0.0/gem.build_complete
/usr/lib64/gems/ruby/pg-1.0.0/pg_ext.so
/usr/share/gems/gems/pg-1.0.0
/usr/share/gems/gems/pg-1.0.0/BSDL
/usr/share/gems/gems/pg-1.0.0/LICENSE
/usr/share/gems/gems/pg-1.0.0/POSTGRES
/usr/share/gems/gems/pg-1.0.0/lib
/usr/share/gems/gems/pg-1.0.0/lib/pg
/usr/share/gems/gems/pg-1.0.0/lib/pg.rb
/usr/share/gems/gems/pg-1.0.0/lib/pg/basic_type_mapping.rb
/usr/share/gems/gems/pg-1.0.0/lib/pg/coder.rb
/usr/share/gems/gems/pg-1.0.0/lib/pg/connection.rb
/usr/share/gems/gems/pg-1.0.0/lib/pg/constants.rb
/usr/share/gems/gems/pg-1.0.0/lib/pg/exceptions.rb
/usr/share/gems/gems/pg-1.0.0/lib/pg/result.rb
/usr/share/gems/gems/pg-1.0.0/lib/pg/text_decoder.rb
/usr/share/gems/gems/pg-1.0.0/lib/pg/text_encoder.rb
/usr/share/gems/gems/pg-1.0.0/lib/pg/type_map_by_column.rb
/usr/share/gems/specifications/pg-1.0.0.gemspec

Compare that to an fpm-generated RPM (truncated for brevity):

$ rpm -ql rubygem-pg-1.4.6-1.x86_64.rpm | head
/usr/lib/.build-id
/usr/lib/.build-id/be
/usr/lib/.build-id/be/dc67914e48db21399c18f5638fdf54590ef15f
/usr/lib/.build-id/be/dc67914e48db21399c18f5638fdf54590ef15f.1
/usr/share/gems/build_info
/usr/share/gems/cache/pg-1.4.6.gem
/usr/share/gems/doc
/usr/share/gems/extensions/x86_64-linux/2.5.0/pg-1.4.6/gem.build_complete
/usr/share/gems/extensions/x86_64-linux/2.5.0/pg-1.4.6/pg/postgresql_lib_path.rb
/usr/share/gems/extensions/x86_64-linux/2.5.0/pg-1.4.6/pg_ext.so

If I add --prefix /usr/lib64, the extensions still don't go to the right place--it's not just a different prefix, it's a different structure.

$ rpm -ql rubygem-pg-1.4.6-1.x86_64.rpm | head
/usr/lib/.build-id
/usr/lib/.build-id/6d
/usr/lib/.build-id/6d/77d4deb1fd26312196f4266271324b7f337a3a
/usr/lib/.build-id/6d/77d4deb1fd26312196f4266271324b7f337a3a.1
/usr/lib64/build_info
/usr/lib64/cache/pg-1.4.6.gem
/usr/lib64/doc
/usr/lib64/extensions/x86_64-linux/2.5.0/pg-1.4.6/gem.build_complete
/usr/lib64/extensions/x86_64-linux/2.5.0/pg-1.4.6/pg/postgresql_lib_path.rb
/usr/lib64/extensions/x86_64-linux/2.5.0/pg-1.4.6/pg_ext.so

Going back to the original error, this is coming from: https://github.com/ruby/ruby/blob/v2_5_9/lib/rubygems/basic_specification.rb#L74 ..due to failing this: https://github.com/ruby/ruby/blob/v2_5_9/lib/rubygems/specification.rb#L2207

I think the relevant check is:

   return false if File.exist? gem_build_complete_path

I added a puts to see what was expected for gem_build_complete_path and got: /usr/lib64/gems/ruby/pg-1.4.6/gem.build_complete

...whereas the file which is actually provided by the package is:

$ rpm -ql rubygem-pg | grep build_complete
/usr/share/gems/extensions/x86_64-linux/2.5.0/pg-1.4.6/gem.build_complete

The building of that path in the ruby code can be tracked to: https://github.com/ruby/ruby/blob/v2_5_9/lib/rubygems/basic_specification.rb#L98

Normally, the call to `Gem.default_ext_dir_for(base_dir)` returns nil:
https://github.com/ruby/ruby/blob/v2_5_9/lib/rubygems/defaults.rb#L62

On AlmaLinux 8.7 (and presumably other RHEL-derived distributions) this is being overridden in:
/usr/share/rubygems/rubygems/defaults/operating_system.rb

I don't have a reference to a git repo for that, but the code is:
def default_ext_dir_for base_dir
  dir = if rpmbuild?
    build_dir = base_dir.chomp Gem.default_dirs[:system][:gem_dir]
    if build_dir != base_dir
      File.join build_dir, Gem.default_dirs[:system][:ext_dir]
    end
  else
    dirs = Gem.default_dirs.detect {|location, paths| paths[:gem_dir] == base_dir}
    dirs && dirs.last[:ext_dir]
  end
  dir && File.join(dir, RbConfig::CONFIG['RUBY_INSTALL_NAME'])
end


This in turn goes into some more disto-specific code in {{operating_system.rb}}, eventually returning `/usr/lib64/gems/ruby`.

I don't know what the best solution is to this; possibly it needs to be possible within fpm to specify a separate extensions path, or maybe it would be possible to use `Gem.extensions_dir` to find the appropriate path.

-Corey