Feature Request: Support Databricks Foundation Model & Pay-Per-Token APIs

[JavOrraca]

Perhaps it's user error but I can't pass a custom OpenAI base_url to redirect the requests to a Databricks serving endpoint. This would be ideal for using {chattr} to interact with Databricks foundation model APIs and pay-per-token LLMs.

Below is an example of how to query these APIs from the Databricks docs:

import os
import openai
from openai import OpenAI

client = OpenAI(
    api_key="dapi-your-databricks-token",
    base_url="https://example.staging.cloud.databricks.com/serving-endpoints"
)

response = client.chat.completions.create(
    model="databricks-dbrx-instruct",
    messages=[
      {
        "role": "system",
        "content": "You are a helpful assistant."
      },
      {
        "role": "user",
        "content": "What is a mixture of experts model?",
      }
    ],
    max_tokens=256
)

Appreciate the help and awesome package!

[edgararuiz]

@zacdav-db - Is this the endpoint that you referred to when we discussed the enhancement to chattr?

[zacdav-db]

@edgararuiz Yep! Haven't yet had a chance to tinker and add it, might do soon.

[zacdav-db]

Begun working on request in #99

[JavOrraca]

Thanks!

[zacdav-db]

This is now merged into main!

[zacdav-db]

This issue can now be closed.