Retrieval Augmented Generation using generative-ai-hub-sdk and HANA vector search

[gianfranco-s]

Tutorials: https://developers.sap.com/tutorials/ai-core-genai-hana-vector.html

Function ask_llm() has too many messages unrelated to what the function is actually doing.

Currently:

def ask_llm(query: str, retrieval_augmented_generation: bool, metric='COSINE_SIMILARITY', k = 4) -> str:

    class color:
        RED = '\033[91m'
        BLUE = '\033[94m'
        BOLD = '\033[1m'
        END = '\033[0m'
    context = ''
    if retrieval_augmented_generation == True:
        print(color.RED + 'Running retrieval augmented generation.' + color.END)
        print(color.RED + '\nEmbedding the query string and running HANA vector search.' + color.END)
        context = run_vector_search(query, metric, k)
        print(color.RED + '\nHANA vector search returned {k} best matching documents.'.format(k=k) + color.END)
        print(color.RED + '\nGenerating LLM prompt using the context information.' + color.END)
    else:
        print(color.RED + 'Generating LLM prompt WITHOUT context information.' + color.END)
    prompt = promptTemplate.format(query=query, context=' '.join(df_context['TEXT'].astype('string')))
    print(color.RED + '\nAsking LLM...' + color.END)
    llm = ChatOpenAI(proxy_model_name='gpt-4', proxy_client=proxy_client)
    response = llm.invoke(prompt).content
    print(color.RED + '...completed.' + color.END)
    print(color.RED + '\nQuery: ' + color.END, query)
    print(color.BLUE + '\nResponse:' + color.BLUE)
    print(response)

Refactored:

def ask_llm(prompt: str,
            proxy_client: GenAIHubProxyClient,
            language_model: str = 'gpt-4') -> str:

    llm = ChatOpenAI(proxy_model_name=language_model.value, proxy_client=proxy_client)
    return llm.invoke(prompt).content

Function call:

query = "I want to calculate a shortest path. How do I do that?"
prompt = promptTemplate.format(query=query, context=' '.join(df_context['TEXT'].astype('string')))
print(prompt)

response = ask_llm(prompt=prompt,
                   proxy_client=proxy_client)
print(response)

[gianfranco-s]

This refactor would modify step 16, because df_context is no longer present inside ask_llm()