Bumblebee error when serving Mistral LLM with latest 0.5.0 release

[brainlid]

Using the latest versions:

• nx: 0.7.0
• exla: 0.7.0
• bumblebee: 0.5.0

Starting a Mistral serving fails:

2024-02-23T18:53:09Z app[4d89d19eb99587] ord [info]** (RuntimeError) conversion failed, expected "sliding_window" to be a number, got: nil
2024-02-23T18:53:09Z app[4d89d19eb99587] ord [info]    (bumblebee 0.5.0) lib/bumblebee/shared/converters.ex:20: anonymous fn/3 in Bumblebee.Shared.Converters.convert!/2
2024-02-23T18:53:09Z app[4d89d19eb99587] ord [info]    (elixir 1.15.7) lib/enum.ex:2510: Enum."-reduce/3-lists^foldl/2-0-"/3
2024-02-23T18:53:09Z app[4d89d19eb99587] ord [info]    (bumblebee 0.5.0) lib/bumblebee/shared/converters.ex:14: Bumblebee.Shared.Converters.convert!/2
2024-02-23T18:53:09Z app[4d89d19eb99587] ord [info]    (bumblebee 0.5.0) lib/bumblebee/text/mistral.ex:389: Bumblebee.HuggingFace.Transformers.Config.Bumblebee.Text.Mistral.load/2
2024-02-23T18:53:09Z app[4d89d19eb99587] ord [info]    (bumblebee 0.5.0) lib/bumblebee.ex:427: Bumblebee.do_load_spec/4
2024-02-23T18:53:09Z app[4d89d19eb99587] ord [info]    (harness 0.1.0) lib/harness/mistral_instruct.ex:12: Harness.MistralInstruct.serving/0
2024-02-23T18:53:09Z app[4d89d19eb99587] ord [info]    (harness 0.1.0) lib/harness/delayed_serving.ex:42: anonymous fn/2 in Harness.DelayedServing.init/1

The project works with the previous versions.

The line of Elixir application code in the stack trace is the Bumblebee.load_spec/2 function call:

    mistral = {:hf, "mistralai/Mistral-7B-Instruct-v0.2"}

    {:ok, spec} =
      Bumblebee.load_spec(mistral,
        module: Bumblebee.Text.Mistral,
        architecture: :for_causal_language_modeling
      )

It appears to be an issue with the Mistral configuration.

[jonatanklosko]

Fixed in 715ad2f. Released as v0.5.1.