ASK AI should work with open source self hosted/cloud hosted LLMs in open source pygwalker

[rishabh-dream11]

Is your feature request related to a problem? Please describe. I am unable to integrate or use Amazon Bedrock, Azure GPT, Gemini or my own llama 2 LLM for the ASK AI feature.

Describe the solution you'd like Either I should have the option to remove the real state used by the ASK AI bar in the open source PyGwalker or I should be able to integrate custom LLM endpoints.

[ObservedObserver]

Thank you for your feedback. Currently, we do not support different llm endpoints or the option to remove the real state used by the ASK AI bar in PyGwalker. We only provide our own API for PyGwalker. While we appreciate your suggestion, we don't have plans to implement these features at the moment.

[rosmur]

since this is open source, we can fork and edit right? could you or someone please provide guidance on which components support this feature?

[rishabh-dream11]

@ObservedObserver Can you keep the issue open for others to submit PR on it.

[ObservedObserver]

Hi @rosmur @rishabh-dream11.

I will keep the issue open and welcome to make PRs. If your are interested implement it, here is how you can develop:

What ask viz does it use LLM to generate a DSL of graphic walker, the DSL's JSON schema can be found at https://graphic-walker.kanaries.net/chartinfo.json

If you want to connect pygwalker with custom LLM, you can build a function (metas, query) => DSL, and pass it to pygwalker. We will add a func parameter for pygwalker in next release which allows you to put your customized function there.

pygwalker by default use Kanaries OpenAPI for generate graphic walker DSL, which is based on GPT-3.5 + RAG. If You want use it to compare with other LLM, you can apply a API token at kanaries website.

[longxiaofei]

pip install --pre --upgrade pygwalker

pygwalker already support custom ask callback function.

from typing import List, Dict, Any

# metas: [{"dragId":"gw_8ffA","fid":"work yes or not","name":"work yes or not","basename":"work yes or not","semanticType":"nominal","analyticType":"dimension"}]
# query: register count per season
# return: refer https://graphic-walker.kanaries.net/chartinfo.json
def ask_callback(metas: List[Dict[str, Any]], query: str) -> Dict[str, Any]:
    return {}

pyg.walk(df, custom_ask_callback=ask_callback)

related pr: #477