IAPark / tiktoken_ruby

Unofficial ruby binding for tiktoken by way of rust
MIT License
125 stars 26 forks source link

Unable to build on OpenBSD #11

Closed basicfeatures closed 1 year ago

basicfeatures commented 1 year ago
$ gem install --user-install tiktoken_ruby
Building native extensions. This could take a while...
ERROR:  Error installing tiktoken_ruby:
        ERROR: Failed to build gem native extension.

    current directory: /home/dev/.local/share/gem/ruby/3.2/gems/tiktoken_ruby-0.0.5/ext/tiktoken_ruby
/usr/local/bin/ruby32 extconf.rb
checking for /usr/bin/clang... yes
checking for /usr/bin/clang++... yes
checking for ar... yes
checking for cargo... yes

current directory: /home/dev/.local/share/gem/ruby/3.2/gems/tiktoken_ruby-0.0.5/ext/tiktoken_ruby
make INSTALL_PROGRAM\=/usr/bin/install\ -c\ \ -s\ -m\ 755 INSTALL_SCRIPT\=/usr/bin/install\ -c\ \ -m\ 755 INSTALL_DATA\=/usr/bin/install\ -c\ \ -m\ 644 INSTALL\=/usr/bin/install\ -c\  DESTDIR\= sitearchdir\=./.gem.20230718-28529-91ilur sitelibdir\=./.gem.20230718-28529-91ilur clean
*** Parse error in /home/dev/.local/share/gem/ruby/3.2/gems/tiktoken_ruby-0.0.5/ext/tiktoken_ruby: Missing dependency operator (Makefile:8)
*** Parse error: Need an operator in 'endif' (Makefile:10)
*** Parse error: Missing dependency operator (Makefile:20)
*** Parse error: Need an operator in 'else' (Makefile:22)
*** Parse error: Need an operator in 'endif' (Makefile:24)
*** Parse error: Missing dependency operator (Makefile:31)
*** Parse error: Need an operator in 'else' (Makefile:33)
*** Parse error: Need an operator in 'endif' (Makefile:35)
*** Parse error: Missing dependency operator (Makefile:265)
*** Parse error: Need an operator in 'endif' (Makefile:267)
*** Parse error: Missing dependency operator (Makefile:523)
*** Parse error: Missing dependency operator (Makefile:528)
*** Parse error: Need an operator in 'endif' (Makefile:530)
*** Parse error: Missing dependency operator (Makefile:534)
*** Parse error: Need an operator in 'else' (Makefile:536)
*** Parse error: Need an operator in 'endif' (Makefile:538)
*** Parse error: Need an operator in 'endif' (Makefile:557)

current directory: /home/dev/.local/share/gem/ruby/3.2/gems/tiktoken_ruby-0.0.5/ext/tiktoken_ruby
make INSTALL_PROGRAM\=/usr/bin/install\ -c\ \ -s\ -m\ 755 INSTALL_SCRIPT\=/usr/bin/install\ -c\ \ -m\ 755 INSTALL_DATA\=/usr/bin/install\ -c\ \ -m\ 644 INSTALL\=/usr/bin/install\ -c\  DESTDIR\= sitearchdir\=./.gem.20230718-28529-91ilur sitelibdir\=./.gem.20230718-28529-91ilur
*** Parse error in /home/dev/.local/share/gem/ruby/3.2/gems/tiktoken_ruby-0.0.5/ext/tiktoken_ruby: Missing dependency operator (Makefile:8)
*** Parse error: Need an operator in 'endif' (Makefile:10)
*** Parse error: Missing dependency operator (Makefile:20)
*** Parse error: Need an operator in 'else' (Makefile:22)
*** Parse error: Need an operator in 'endif' (Makefile:24)
*** Parse error: Missing dependency operator (Makefile:31)
*** Parse error: Need an operator in 'else' (Makefile:33)
*** Parse error: Need an operator in 'endif' (Makefile:35)
*** Parse error: Missing dependency operator (Makefile:265)
*** Parse error: Need an operator in 'endif' (Makefile:267)
*** Parse error: Missing dependency operator (Makefile:523)
*** Parse error: Missing dependency operator (Makefile:528)
*** Parse error: Need an operator in 'endif' (Makefile:530)
*** Parse error: Missing dependency operator (Makefile:534)
*** Parse error: Need an operator in 'else' (Makefile:536)
*** Parse error: Need an operator in 'endif' (Makefile:538)
*** Parse error: Need an operator in 'endif' (Makefile:557)

make failed, exit code 1

Gem files will remain installed in /home/dev/.local/share/gem/ruby/3.2/gems/tiktoken_ruby-0.0.5 for inspection.
Results logged to /home/dev/.local/share/gem/ruby/3.2/extensions/x86_64-openbsd/3.2/tiktoken_ruby-0.0.5/gem_make.out

I see it's aware of OpenBSD's clang and clang++ which is great. For those who don't know, OpenBSD's has its own rewrite alternative of almost everything, including make.

$ env
_=/usr/bin/env
LOGNAME=dev
HOME=/home/dev
SSH_TTY=/dev/ttyp0
GEM_PATH=/home/dev/.bundle/ruby/3.2:/home/dev/.local/share/gem/ruby/3.2:/usr/local/lib/ruby/gems/3.2
CXX=/usr/bin/clang++
SHELL=/bin/ksh
GEM_HOME=/home/dev/.bundle/ruby/3.2:/home/dev/.local/share/gem/ruby/3.2
PWD=/home/dev
CC=/usr/bin/clang
PATH=/bin:/sbin:/usr/bin:/usr/sbin:/usr/X11R6/bin:/usr/local/bin:/usr/local/sbin:/home/dev/.bundle/ruby/3.2/bin:/home/dev/.local/share/gem/ruby/3.2/bin
$ gem env
RubyGems Environment:
  - RUBYGEMS VERSION: 3.4.10
  - RUBY VERSION: 3.2.2 (2023-03-30 patchlevel 53) [x86_64-openbsd]
  - INSTALLATION DIRECTORY: /home/dev/.bundle/ruby/3.2:/home/dev/.local/share/gem/ruby/3.2
  - USER INSTALLATION DIRECTORY: /home/dev/.local/share/gem/ruby/3.2
  - RUBY EXECUTABLE: /usr/local/bin/ruby32
  - GIT EXECUTABLE: /usr/local/bin/git
  - EXECUTABLE DIRECTORY: /home/dev/.bundle/ruby/3.2:/home/dev/.local/share/gem/ruby/3.2/bin
  - SPEC CACHE DIRECTORY: /home/dev/.local/share/gem/specs
  - SYSTEM CONFIGURATION DIRECTORY: /etc
  - RUBYGEMS PLATFORMS:
     - ruby
     - x86_64-openbsd
  - GEM PATHS:
     - /home/dev/.bundle/ruby/3.2:/home/dev/.local/share/gem/ruby/3.2
     - /home/dev/.bundle/ruby/3.2
     - /home/dev/.local/share/gem/ruby/3.2
     - /usr/local/lib/ruby/gems/3.2
  - GEM CONFIGURATION:
     - :update_sources => true
     - :verbose => true
     - :backtrace => true
     - :bulk_threshold => 1000
  - REMOTE SOURCES:
     - https://rubygems.org/
  - SHELL PATH:
     - /bin
     - /sbin
     - /usr/bin
     - /usr/sbin
     - /usr/X11R6/bin
     - /usr/local/bin
     - /usr/local/sbin
     - /home/dev/.bundle/ruby/3.2/bin
     - /home/dev/.local/share/gem/ruby/3.2/bin

Anybody know what to do?

basicfeatures commented 1 year ago

pkg_add llvm seemed to do the trick.

basicfeatures commented 1 year ago

Actually that was not at all how. But I found a temp workaround, maybe leave this issue open until we have a real solution?

$ ruby -v
ruby 3.2.2 (2023-03-30 revision e51014f9c0) [x86_64-openbsd]
$ uname -sr
OpenBSD 7.3

$ gem install --user-install tiktoken_ruby
[...fails...]
$ doas pkg_add gmake
$ export LIBCLANG_PATH=/usr/local/lib/
$ gmake
$ doas gmake install

Nasty!

basicfeatures commented 1 year ago

Found the way:

make=gmake LIBCLANG_PATH=/usr/local/lib gem install --user-install tiktoken_ruby