search

elixir-nx / bumblebee

Pre-trained Neural Network models in Axon (+ 🤗 Models integration)
Apache License 2.0
1.33k stars 95 forks source link

Bumblebee fails to load gpt models in main. #293

Closed ityonemo closed 10 months ago

ityonemo commented 10 months ago

In bumblebee main (73c6e5f5d90aa625eda7247e60c2f63198a5c154), the following code is sufficient to fail:

Nx.default_backend({EXLA.Backend, client: :host})
auth_token = System.get_env("HF_AUTH_TOKEN")
{:ok, model} = Bumblebee.load_model({:hf, "meta-llama/Llama-2-7b-chat-hf", auth_token: auth_token})

with the following error:

** (RuntimeError) conversion failed, invalid format for "rope_scaling", got: nil

same code works with "~> 0.4.2".

nx is pinned to 0.6

jonatanklosko commented 10 months ago

Fixed in 3cc0ff9d9a57839a249af81f24d0b58f6b56261d.