[Feature Request]: Supporting routing via Not Diamond

[acompa]

Feature Description

Hi llama_index folks! As a fan of your library, I want to contribute a routing integration supported by Not Diamond.*

Our API works similarly to your standalone selector functionality. We expect to route prompts across multiple indexes, though, similar to PydanticMultiSelector.

That said, I also see that other routing options implemented as contributing LLMs. See OpenRouter and Unify for examples.

I have some questions before starting our contribution:

• Do you recommend I implement a MultiSelector, or proceed with a LLM like other routers?
• If we go with a selector, do you agree that adding a NotDiamondQueryEngine to llama-index-integrations would make sense for ND users seeking a ND-powered query engine?
• Relatedly, I noticedllama-index-integrations has neither query engine nor selector integrations. What are your thoughts on adding these modules as part of this contribution?

Thanks for fielding these questions - really looking forward to adding our routing to llama_index!!

---

* _Briefly, on us: we're offering AI-powered routing for LLM prompts, so that users can focus on workflows and prompts instead of integrating with the latest LLM releases or various provider libraries. You can watch Tomas walk through our SDK here._

Reason

Time, implementation bandwidth etc

Value of Feature

Routing selection via Not Diamond can help llama_index users benefit from both higher prompt accuracy and lower cost.

[acompa]

Okay, I believe I want to create a NotDiamondRouterQueryEngine, which folks can then call via

notdiamond = NotDiamondQueryEngine()
messages = [...]
response = notdiamond.query(messages)
print(str(response.metadata["selector_result"]))

More details:

• This example notebook walks through a clear query pattern.
• RouterQueryEngine code.
  • This takes a BaseSelector, which has abstract methods _select and _aselect which I probably want to implement.
  • Answering an earlier q: I probably want a SingleSelector, since ND only returns single model recommendations.
• I didn't know LlamaIndex supports custom query engines but I don't think we actually want to create one!

[acompa]

Changed my implementation slightly towards defining a NotDiamondSelector which folks can use within a RouterQueryEngine or as a standalone selector. Details available in the linked PR above ^^.