Databricks API support

[nlpravi]

Do you support Databricks foundation model APIs?

https://docs.databricks.com/en/machine-learning/foundation-models/index.html

Thanks, Ravi.

[ishaan-jaff]

Hi @nlpravi this looks OpenAI compatible - so we should already support this, can you try if this works ?

Is this what you were looking for ?

import os 
from litellm import completion
response = completion(
    model = "openai/databricks-llama-2-70b-chat",
    api_key="api_key="dapi-your-databricks-token",
    api_base="https://example.staging.cloud.databricks.com/serving-endpoints", 
    messages=[{ "content": "Hello, how are you?","role": "user"}]
)

[krrishdholakia]

Closing this as a no-op, but please reopen if the above snippet doesn't work for you or there's something databricks specific we need to do for this - @nlpravi

[ishaan-jaff]

@nlpravi wanted to follow up did you want us to support databricks in the same way we support Groq AI ?

https://github.com/BerriAI/litellm/pull/2168 ?

[nlpravi]

Yes, that would be great.

Thank you.

On Sat, Feb 24, 2024 at 11:21 AM Ishaan Jaff @.***> wrote:

@nlpravi https://github.com/nlpravi wanted to follow up did you want us to support databricks in the same way we support Groq AI ?

2168 https://github.com/BerriAI/litellm/pull/2168 ?

— Reply to this email directly, view it on GitHub https://github.com/BerriAI/litellm/issues/2160#issuecomment-1962427987, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAOY3CKWEEERMNLADS3WP5DYVIOS3AVCNFSM6AAAAABDW43XBWVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSNRSGQZDOOJYG4 . You are receiving this because you were mentioned.Message ID: @.***>

-- Ravi

[ishaan-jaff]

re-opening since user said a great solution here would be to support the way we support Groq AI

[krrishdholakia]

What are the relevant keys for this? @nlpravi

[nlpravi]

Thank you Krish. You need a Databricks PAT (Personal Access Token) to access this API. I'm not sure how to create a PAT in the community (free) edition. I have access to the enterprise edition. This PAT will serve as the API token to access their foundation model APIs.

The REST API reference is here Foundation Model REST API reference | Databricks on AWS https://docs.databricks.com/en/machine-learning/foundation-models/api-reference.html

To call the APIs you need to set up a workspace in your account and have a key (access token).

Thanks, Ravi.

https://docs.databricks.com/en/machine-learning/foundation-models/api-reference.html

On Sat, Feb 24, 2024 at 4:24 PM Krish Dholakia @.***> wrote:

What are the relevant keys for this? @nlpravi https://github.com/nlpravi

— Reply to this email directly, view it on GitHub https://github.com/BerriAI/litellm/issues/2160#issuecomment-1962746047, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAOY3COLNNALGGLMZNXBPUDYVJSCDAVCNFSM6AAAAABDW43XBWVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSNRSG42DMMBUG4 . You are receiving this because you were mentioned.Message ID: @.***>

-- Ravi

[nlpravi]

Here is how you can call Databricks API through langchain. You need a workspace and the databricks token

import os

os.environ['DATABRICKS_HOST']="https://.cloud.databricks.com"

os.environ['DATABRICKS_TOKEN']="dapixxxxxxxxxxxxxxxxxx"

from langchain.chat_models import ChatDatabricks

from langchain_core.messages import HumanMessage, SystemMessage

messages = [

SystemMessage(content="You're a helpful assistant"),

HumanMessage(content="Can you explain AI in ten words?"),

]

chat_model = ChatDatabricks(endpoint="databricks-mixtral-8x7b-instruct", max_tokens=500)

print(chat_model.invoke(messages))

On Sun, Feb 25, 2024 at 11:42 AM Ravi Kondadadi @.***> wrote:

Thank you Krish. You need a Databricks PAT (Personal Access Token) to access this API. I'm not sure how to create a PAT in the community (free) edition. I have access to the enterprise edition. This PAT will serve as the API token to access their foundation model APIs.

The REST API reference is here Foundation Model REST API reference | Databricks on AWS https://docs.databricks.com/en/machine-learning/foundation-models/api-reference.html

To call the APIs you need to set up a workspace in your account and have a key (access token).

Thanks, Ravi.

https://docs.databricks.com/en/machine-learning/foundation-models/api-reference.html

On Sat, Feb 24, 2024 at 4:24 PM Krish Dholakia @.***> wrote:

What are the relevant keys for this? @nlpravi https://github.com/nlpravi

— Reply to this email directly, view it on GitHub https://github.com/BerriAI/litellm/issues/2160#issuecomment-1962746047, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAOY3COLNNALGGLMZNXBPUDYVJSCDAVCNFSM6AAAAABDW43XBWVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSNRSG42DMMBUG4 . You are receiving this because you were mentioned.Message ID: @.***>

-- Ravi

-- Ravi

[krrishdholakia]

+1 second user asked for this

[demux79]

DBRX-Instruct is really powerful. I would love to be able to use it as well. Your help is highly appreciated!

[krrishdholakia]

picking this up today