[1] DatabricksRM: Use databricks-sdk to fetch token / workspace URL + several small improvements

[dbczumar]

DatabricksRM: Use databricks-sdk to fetch token / workspace URL + several small improvements

[okhat]

@krypticmouse Any idea if the docs here are easy to fix before we merge?

[krypticmouse]

@okhat @dbczumar The issue is in docs/dspy-usecases.md file line 112, the link markdown is wrong:

https://github.com/stanfordnlp/dspy/blob/cdbea6eb9accf1f1d2cc3f8ce333502bddf3b8fb/docs/docs/dspy-usecases.md?plain=1#L112C1-L112C143

Change line 112 from this:

| **Langfuse** | [Link]([https://docs.langtrace.ai/supported-integrations/llm-frameworks/dspy](https://langfuse.com/docs/integrations/dspy)) |

To this:

| **Langfuse** | [Link](https://langfuse.com/docs/integrations/dspy) |

[okhat]

@arnavsinghvi11 is just checking if this will break any existing documentation on the DB site etc since the changes affect the interface.

[krypticmouse]

If the build fails vercel won’t merge it so there would be no changes reflected in the existing website.

[arnavsinghvi11]

Thanks @dbczumar for the updates! All looks good.

With the changes regarding https://github.com/stanfordnlp/dspy/pull/1564/files#r1780356012, can we update the DSPy on Databricks blog post to reflect this?

Specifically, how the retriever is configured and the retreived_results is called through DatabricksRM

retriever = DatabricksRM(
    databricks_index_name = "your_index_name",
    docs_id_column_name="id",
    text_column_name="field2",
    k=3
)
retrieved_results = DatabricksRM(query="Example query text", query_type="hybrid"))

[dbczumar]

Thanks @arnavsinghvi11, absolutely! I'll get that blog updated. I'll also file a follow-up PR with some test coverage, but I've QAed this PR manually