search

IAPark / tiktoken_ruby

Unofficial ruby binding for tiktoken by way of rust
MIT License
109 stars 26 forks source link

Getting "LoadError" when trying to use the gem as dependency #10

Open rolo9128 opened 1 year ago

rolo9128 commented 1 year ago

I'm trying to use langchainrb gem, which has tiktoken_rb as a dependency. I'm getting a LoadError when trying to use the gem:

LoadError - cannot load such file -- /gems/ruby/3.0.0/gems/tiktoken_ruby-0.0.5-x86_64-linux-musl/lib/tiktoken_ruby/tiktoken_ruby:
 |   tiktoken_ruby-0.0.5-x86_64-linux (musl) lib/tiktoken_ruby.rb:10:in `require_relative'
 |   tiktoken_ruby-0.0.5-x86_64-linux (musl) lib/tiktoken_ruby.rb:10:in `rescue in <top (required)>'
 |   tiktoken_ruby-0.0.5-x86_64-linux (musl) lib/tiktoken_ruby.rb:6:in `<top (required)>'

Is this something I can get help with? Thanks!

weilandia commented 5 months ago

I had the same issue. Ended up forking langchainrb and removing the tiktoken dep

andreibondarev commented 1 month ago

@weilandia I had gone ahead and demoted tiktoken to an optional dep: https://github.com/patterns-ai-core/langchainrb/pull/649

weilandia commented 2 weeks ago

I think this is actually fixed on x86_64-linux so no longer an issue, but nice to be optional.