ankane / tokenizers-ruby

Fast state-of-the-art tokenizers for Ruby
Apache License 2.0
132 stars 6 forks source link

Unable to run Rails when gem is installed #10

Closed marckohlbrugge closed 1 year ago

marckohlbrugge commented 1 year ago

After successfully installing the gem (tokenizers (0.2.2-arm64-darwin)), I'm unable to open Rails console:

➜ rails console
[1]    8415 killed     rails console

When trying to install HEAD version, I get this error message:

Gem::Ext::BuildError: ERROR: Failed to build gem native extension.
…snip…
thread 'main' panicked at 'rustfmt is required to generate bindings. To install, run `rustup component add rustfmt`: Os { code: 2, kind: NotFound, message: "No such file or directory"
}', /Users/marc/.cargo/registry/src/github.com-1ecc6299db9ec823/rb-sys-build-0.9.56/src/bindings.rs:77:10
  note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
warning: build failed, waiting for other jobs to finish...
make: *** [target/release/libtokenizers.dylib] Error 101
ankane commented 1 year ago

Hey @marckohlbrugge, thanks for the report. Looks like this is a similar issue to Polars Ruby (https://github.com/ankane/polars-ruby/issues/2).

Are there any crash reports in the Console? https://support.apple.com/guide/console/reports-cnsl664be99a/mac

marckohlbrugge commented 1 year ago

I've shared a crash log with you privately via Twitter DM.

Looks like it might be a code signing issue?

Here's a summary of the log:

-------------------------------------
Translated Report (Full Report Below)
-------------------------------------

Process:               ruby [38389]
Path:                  /Users/USER/*/ruby
Identifier:            ruby
Version:               ???
Code Type:             ARM-64 (Native)
Parent Process:        zsh [21260]
Responsible:           iTerm2 [618]
User ID:               501

Date/Time:             2023-01-22 10:57:50.6255 +0700
OS Version:            macOS 13.1 (22C65)
Report Version:        12
Anonymous UUID:        [removed]

Sleep/Wake UUID:       [removed]

Time Awake Since Boot: 130000 seconds
Time Since Wake:       2259 seconds

System Integrity Protection: enabled

Crashed Thread:        0  Dispatch queue: com.apple.main-thread

Exception Type:        EXC_BAD_ACCESS (SIGKILL (Code Signature Invalid))
Exception Codes:       UNKNOWN_0x32 at 0x000000010381c000
Exception Codes:       0x0000000000000032, 0x000000010381c000

Termination Reason:    Namespace CODESIGNING, Code 2 Invalid Page
marckohlbrugge commented 1 year ago

Not sure if this is the correct way to verify a code signature, but it does say invalid signature

codesign --verify /Users/marc/.asdf/installs/ruby/3.0.3/lib/ruby/gems/3.0.0/gems/tokenizers-0.2.2-arm64-darwin/lib/tokenizers/3.0/tokenizers.bundle
/Users/marc/.asdf/installs/ruby/3.0.3/lib/ruby/gems/3.0.0/gems/tokenizers-0.2.2-arm64-darwin/lib/tokenizers/3.0/tokenizers.bundle: invalid signature (code or signature have been modified)
In architecture: arm64
marckohlbrugge commented 1 year ago

Manually replacing the signature allowed me to use tokenizers! Of course it's not a proper solution, but it does confirm it's a code signing issue.

sudo codesign -f -s - /Users/marc/.asdf/installs/ruby/3.0.3/lib/ruby/gems/3.0.0/gems/tokenizers-0.2.2-arm64-darwin/lib/tokenizers/3.0/tokenizers.bundle
Password:
/Users/marc/.asdf/installs/ruby/3.0.3/lib/ruby/gems/3.0.0/gems/tokenizers-0.2.2-arm64-darwin/lib/tokenizers/3.0/tokenizers.bundle: replacing existing signature
ankane commented 1 year ago

Awesome, thanks @marckohlbrugge! I think it's probably due to stripping symbols and debuginfo, so either need to disable that or re-sign afterwards. Will fix and try to get a release out tomorrow.

ankane commented 1 year ago

New version is out with a valid signature. Thanks again!