Binary not fetched in gem install

[danielpclark]

Steps to reproduce:

docker run -i -t ruby /bin/bash -l
gem install faster_path

Output:

Fetching: minitar-0.6.1.gem (100%)
The `minitar` executable is no longer bundled with `minitar`. If you are
expecting this executable, make sure you also install `minitar-cli`.
Successfully installed minitar-0.6.1
Fetching: tomlrb-1.2.6.gem (100%)
Successfully installed tomlrb-1.2.6
Fetching: thermite-0.13.0.gem (100%)
Successfully installed thermite-0.13.0
Fetching: faster_path-0.3.2.gem (100%)
Building native extensions. This could take a while...
ERROR:  Error installing faster_path:
    ERROR: Failed to build gem native extension.

    current directory: /usr/local/bundle/gems/faster_path-0.3.2/ext
/usr/local/bin/ruby -rrubygems /usr/local/lib/ruby/gems/2.5.0/gems/rake-12.3.0/exe/rake RUBYARCHDIR=/usr/local/bundle/extensions/x86_64-linux/2.5.0/faster_path-0.3.2 RUBYLIBDIR=/usr/local/bundle/extensions/x86_64-linux/2.5.0/faster_path-0.3.2
checking for cargo... no
rake aborted!
****
Rust's Cargo is required to build this extension. Please install
Rust and put it in the PATH, or set the CARGO environment variable appropriately.
****
/usr/local/bundle/gems/thermite-0.13.0/lib/thermite/cargo.rb:77:in `inform_user_about_cargo'
/usr/local/bundle/gems/thermite-0.13.0/lib/thermite/tasks.rb:128:in `block in define_build_task'
Tasks: TOP => default => thermite:build
(See full trace by running task with --trace)
Downloading compiled version (0.0.1) from GitHub

rake failed, exit code 1

Gem files will remain installed in /usr/local/bundle/gems/faster_path-0.3.2 for inspection.
Results logged to /usr/local/bundle/extensions/x86_64-linux/2.5.0/faster_path-0.3.2/gem_make.out

[malept]

It looks like it's because your Cargo version isn't in sync with the gem version. I suggest setting the version in the gemspec to the Cargo.toml value:

https://github.com/data-axle/t12r/blob/5704dc253b5452626dd5858a93aa2bc1898c5a2a/t12r.gemspec#L4

For future reference, it might be useful to look at the thermite logs to see what's going on: https://github.com/malept/thermite/blob/v0.13.0/README.md#troubleshooting - I can always add more logging if it's helpful.

[danielpclark]

I'm not sure I want to do that. I've been planning to have a Rust crate also from the same project. Once I've reached a 1.0 for FasterPath I plan on having the Rust type faster_path::Pathname available with the same convenience methods as what Ruby gets in Pathname.

I've only reserved the rust crate name faster_path at 0.0.1 on crates.io for now until I get to it.

[malept]

How about a different tag for Rust releases, then? That is what I have configured in https://github.com/malept/rusty_blank (although I'm not sure if github_release_type = "latest" is what you want).

[danielpclark]

Doesn't the Github Releases provide the overall category of tag? Like v0.3.2 is the Releases number even if it's not in the tarball name.

[danielpclark]

I would really appreciate it if you re-opened the issue. I've taken the time to figure out the steps to fix this. Here's what it looks like in a simple script form:

require 'net/http'
require 'rexml/document'
require 'uri'
require 'json'

spec = Gem::Specification::load("faster_path.gemspec")

GH_API = "https://api.github.com/repos"
REPO = "danielpclark/faster_path"
VERSION = "v#{spec.version}"

response = Net::HTTP.get_response(URI("#{GH_API}/#{REPO}/releases/tags/#{VERSION}"))
data = JSON.parse(respons.body)
links = data["assets"].map {|h| [h.dig("name"), h.dig("browser_download_url")] }.to_h

pp links.keys
# ["faster_path-0.0.1-ruby23-darwin16-x86_64.tar.gz",
#  "faster_path-0.0.1-ruby23-linux-x86_64.tar.gz",
#  "faster_path-0.0.1-ruby24-darwin16-x86_64.tar.gz",
#  "faster_path-0.0.1-ruby24-linux-x86_64.tar.gz",
#  "faster_path-0.0.1-ruby25-darwin16-x86_64.tar.gz",
#  "faster_path-0.0.1-ruby25-linux-x86_64.tar.gz"]

GEM = spec.name
LIB_VERSION = File.open("Cargo.toml").readlines.grep(/version = "(\d+\.\d+\.\d+)"/).first.match(/"(.{5,})"/)[1]
# Note: I prefer the full semantic version of Ruby in the file name with patch level
RUBY = "ruby#{RUBY_VERSION}"
OS = RbConfig::CONFIG['target_os']
ARCH = RbConfig::CONFIG['target_cpu']

download_file = "#{GEM}-#{LIB_VERSION}-#{RUBY}-#{OS}-#{ARCH}.tar.gz"
download_url = links[download_file]
puts download_url
# => "https://github.com/danielpclark/faster_path/releases/download/v0.3.2/faster_path-0.0.1-ruby25-linux-x86_64.tar.gz"

Consider this a feature request. I'd like it to work with this file structure and versioning systems. You don't need to change the gem version lookup or toml reading… this is only a working proof of concept.

[danielpclark]

On second thought I think I've figured out a way to make it work for my particular project. So It's up to you whether you'd like to add the design I've requested.

[danielpclark]

Well I still haven't been able to get it to work with gem install in docker without cargo. So I went ahead and tried it for two other projects; the one you listed above t12r and rusty_blank and neither of them work.

FasterPath

Fetching: faster_path-0.3.5.gem (100%)
Building native extensions. This could take a while...
ERROR:  Error installing faster_path:
    ERROR: Failed to build gem native extension.

    current directory: /usr/local/bundle/gems/faster_path-0.3.5/ext
/usr/local/bin/ruby -rrubygems /usr/local/bundle/gems/rake-12.3.0/exe/rake RUBYARCHDIR=/usr/local/bundle/extensions/x86_64-linux/2.3.0/faster_path-0.3.5 RUBYLIBDIR=/usr/local/bundle/extensions/x86_64-linux/2.3.0/faster_path-0.3.5
checking for cargo... no
rake aborted!
****
Rust's Cargo is required to build this extension. Please install
Rust and put it in the PATH, or set the CARGO environment variable appropriately.
****
/usr/local/bundle/gems/thermite-0.13.0/lib/thermite/cargo.rb:77:in `inform_user_about_cargo'
/usr/local/bundle/gems/thermite-0.13.0/lib/thermite/tasks.rb:128:in `block in define_build_task'
/usr/local/bundle/gems/rake-12.3.0/exe/rake:27:in `<main>'
Tasks: TOP => default => thermite:build
(See full trace by running task with --trace)
Downloading compiled version (0.0.1) from GitHub

rake failed, exit code 1

Gem files will remain installed in /usr/local/bundle/gems/faster_path-0.3.5 for inspection.
Results logged to /usr/local/bundle/extensions/x86_64-linux/2.3.0/faster_path-0.3.5/gem_make.out

t12r

After downloading the zip of the repo and doing a gem build then gem install:

gem install t12r-0.1.7.gem 
Building native extensions. This could take a while...
ERROR:  Error installing t12r-0.1.7.gem:
    ERROR: Failed to build gem native extension.

    current directory: /usr/local/bundle/gems/t12r-0.1.7/ext
/usr/local/bin/ruby -rrubygems /usr/local/bundle/gems/rake-12.3.0/exe/rake RUBYARCHDIR=/usr/local/bundle/extensions/x86_64-linux/2.3.0/t12r-0.1.7 RUBYLIBDIR=/usr/local/bundle/extensions/x86_64-linux/2.3.0/t12r-0.1.7
rake aborted!
Errno::ENOENT: No such file or directory @ rb_sysopen - /usr/local/bundle/gems/t12r-0.1.7/Cargo.toml
/usr/local/bundle/gems/tomlrb-1.2.6/lib/tomlrb.rb:41:in `read'
/usr/local/bundle/gems/tomlrb-1.2.6/lib/tomlrb.rb:41:in `load_file'
/usr/local/bundle/gems/thermite-0.13.0/lib/thermite/config.rb:239:in `toml'
/usr/local/bundle/gems/thermite-0.13.0/lib/thermite/config.rb:255:in `toml_config'
/usr/local/bundle/gems/thermite-0.13.0/lib/thermite/tasks.rb:109:in `initialize'
/usr/local/bundle/gems/t12r-0.1.7/ext/Rakefile:18:in `new'
/usr/local/bundle/gems/t12r-0.1.7/ext/Rakefile:18:in `<top (required)>'
/usr/local/bundle/gems/rake-12.3.0/exe/rake:27:in `<main>'
(See full trace by running task with --trace)

rake failed, exit code 1

Gem files will remain installed in /usr/local/bundle/gems/t12r-0.1.7 for inspection.
Results logged to /usr/local/bundle/extensions/x86_64-linux/2.3.0/t12r-0.1.7/gem_make.out

rusty_blank

Same steps as above

gem install rusty_blank-0.0.3.gem 
Building native extensions. This could take a while...
ERROR:  Error installing rusty_blank-0.0.3.gem:
    ERROR: Failed to build gem native extension.

    current directory: /usr/local/bundle/gems/rusty_blank-0.0.3
/usr/local/bin/ruby -rrubygems /usr/local/bundle/gems/rake-12.3.0/exe/rake RUBYARCHDIR=/usr/local/bundle/extensions/x86_64-linux/2.3.0/rusty_blank-0.0.3 RUBYLIBDIR=/usr/local/bundle/extensions/x86_64-linux/2.3.0/rusty_blank-0.0.3
rake aborted!
Unable to determine name from existing gemspec. Use :name => 'gemname' in #install_tasks to manually set it.
/usr/local/bundle/gems/rusty_blank-0.0.3/Rakefile:1:in `<top (required)>'
/usr/local/bundle/gems/rake-12.3.0/exe/rake:27:in `<main>'
(See full trace by running task with --trace)

rake failed, exit code 1

Gem files will remain installed in /usr/local/bundle/gems/rusty_blank-0.0.3 for inspection.
Results logged to /usr/local/bundle/extensions/x86_64-linux/2.3.0/rusty_blank-0.0.3/gem_make.out

My project made it further than the other two. I suppose their gemspecs may not have included some important required files. I'm having trouble proving your Github Releases feature can get the binaries.

[danielpclark]

Okay. I finally got it. You may ignore my ranting. Thanks again for all your hard work!

[malept]

My thought is to make the "download from GitHub Releases" feature more flexible so that someone can make an extension to support the "Gem version in tag but Cargo version in filename" use case. I don't particularly want it in the core right now because the code in that feature is getting unwieldy as it is. I'll file a new issue detailing what I think it should look like soon.

[danielpclark]

I don't particularly want it in the core right now because the code in that feature is getting unwieldy as it is.

With that I can agree. Your code organization is wonderful but the heavy coupling across all contexts make for the code to be very hard to read, especially from an outsider. If I wasn't a pro at Ruby I don't think I would be able to understand it and would likely have given up trying.

Here's what I did to make my system work the way I liked. I put the following in a file named thermite_initialization and then included it after ever other inclusion of Thermite by require.

require 'rubygems'
require_relative './version'

module Thermite
  class Config
    def ruby_version
      "ruby#{RUBY_VERSION}"
    end
  end

  class Tasks
    def github_download_uri(_tag, version)
      "#{github_uri}/releases/download/v#{FasterPath::VERSION}/#{config.tarball_filename(version)}"
    end
  end
end

On refactoring options I would recommend removing all coupling (of modules needing to know how each other worked) and have methods changed to functional style with A input and B output with no side effects (doing anything other than producing B). This would make it have both readability and composability.

[danielpclark]

I found a video clip that expresses perfectly by what I meant when I said “heavy coupling”.