search

thmsmlr / instructor_ex

Structured outputs for LLMs in Elixir
https://hexdocs.pm/instructor
520 stars 58 forks source link

readme

instructor_ex

Structured, Ecto outputs with OpenAI (and OSS LLMs)

Instructor version Hex Docs Hex Downloads GitHub stars Twitter Follow Discord

Structured prompting for LLMs. Instructor is a spiritual port of the great Instructor Python Library by @jxnlco, check out his talk on YouTube.

The Instructor library is useful for coaxing an LLM to return JSON that maps to an Ecto schema that you provide, rather than the default unstructured text output. If you define your own validation logic, Instructor can automatically retry prompts when validation fails (returning natural language error messages to the LLM, to guide it when making corrections).

Instructor is designed to be used with the OpenAI API by default, but it also works with llama.cpp and Bumblebee (Coming Soon!) by using an extendable adapter behavior.

At its simplest, usage is pretty straightforward:

  1. Create an ecto schema, with a @doc string that explains the schema definition to the LLM.
  2. Define a validate_changeset/1 function on the schema, and use the Instructor.Validator macro in order for Instructor to know about it.
  3. Make a call to Instructor.chat_completion/1 with an instruction for the LLM to execute.

You can use the max_retries parameter to automatically, iteratively go back and forth with the LLM to try fixing validation errorswhen they occur.

defmodule SpamPrediction do
  use Ecto.Schema
  use Instructor.Validator

  @doc """
  ## Field Descriptions:
  - class: Whether or not the email is spam.
  - reason: A short, less than 10 word rationalization for the classification.
  - score: A confidence score between 0.0 and 1.0 for the classification.
  """
  @primary_key false
  embedded_schema do
    field(:class, Ecto.Enum, values: [:spam, :not_spam])
    field(:reason, :string)
    field(:score, :float)
  end

  @impl true
  def validate_changeset(changeset) do
    changeset
    |> Ecto.Changeset.validate_number(:score,
      greater_than_or_equal_to: 0.0,
      less_than_or_equal_to: 1.0
    )
  end
end

is_spam? = fn text ->
  Instructor.chat_completion(
    model: "gpt-3.5-turbo",
    response_model: SpamPrediction,
    max_retries: 3,
    messages: [
      %{
        role: "user",
        content: """
        Your purpose is to classify customer support emails as either spam or not.
        This is for a clothing retail business.
        They sell all types of clothing.

        Classify the following email: 
    #{text}
    ```
    """
  }
]

) end

is_spam?.("Hello I am a Nigerian prince and I would like to send you money")

=> {:ok, %SpamPrediction{class: :spam, reason: "Nigerian prince email scam", score: 0.98}}


Check out our [Quickstart Guide](https://hexdocs.pm/instructor/quickstart.html) for more code snippets that you can run locally (in Livebook). Or, to get a better idea of the thinking behind Instructor, read more about our [Philosophy & Motivations](https://hexdocs.pm/instructor/philosophy.html).

Optionally, you can also customize the your llama.cpp calls (with defaults shown):
```elixir
llamacpp
config :instructor, adapter: Instructor.Adapters.Llamacpp
config :instructor, :llamacpp,
    chat_template: :mistral_instruct,
    api_url: "http://localhost:8080/completion"

Installation

In your mix.exs,

def deps do
  [
    {:instructor, "~> 0.0.5"}
  ]
end

InstructorEx uses Code.fetch_docs/1 to fetch LLM instructions from the Ecto schema specified in response_model. If your project is deployed using releases, add the following configuration to mix.exs to prevent docs from being stripped from the release:

def project do
  # ...
  releases: [
    myapp: [
      strip_beams: [keep: ["Docs"]]
    ]
  ]
end

TODO

Blog Posts