page_type: sample languages:
- azdeveloper
- java
- bicep
- typescript
- html products:
- azure
- azure-openai
- active-directory
- azure-cognitive-search
- azure-app-service
- azure-container-apps
- azure-kubernetes-service
- azure-sdks
- github
- document-intelligence
- azure-monitor
-
azure-pipelines urlFragment: azure-search-openai-demo-java name: ChatGPT + Enterprise data (Java) on App Service, Azure Container Apps, and Azure Kubernetes Service description: A Java sample app that chats with your data using OpenAI and AI Search.
ChatGPT + Enterprise data with Azure OpenAI and Azure AI Search - Java Version
This repo is the Java version of the well known ChatGPT + Enterprise data code sample originally written in python.
It demonstrates best practices for creating ChatGPT-like experiences over your own data using the Retrieval Augmented Generation pattern. It uses Azure OpenAI Service to access the ChatGPT model gpt-35-turbo, and Azure Azure AI Search for data indexing and retrieval.
This repository includes sample data so it's ready to try end to end. In this sample application we use a fictitious company called Contoso Electronics, and the experience allows its employees to ask questions about the benefits, internal policies, as well as job descriptions and roles.
What this demo application does:
- Chat and Q&A interfaces
- Explores various options to help users evaluate the trustworthiness of responses with citations, tracking of source content, etc.
- Shows possible approaches for data preparation, prompt construction, and orchestration of interaction between model (ChatGPT) and retriever (Azure AI Search)
- Shows possible AI orchestration implementation using the plain Java Open AI sdk or the Java Semantic Kernel sdk
- Settings directly in the UX to tweak the behavior and experiment with options
Solution architecture and deployment options
This sample supports different architectural styles. It can be deployed as standalone app on top of Azure App Service or as a microservice event driven architecture with web frontend, AI orchestration and document ingestion apps hosted by Azure Container Apps or Azure Kubernetes Service.
- For Azure App Service deployment, see here.
- For Azure Container Apps deployment, see here.
- For Azure Kubernetes Service deployment, see here.
RAG implementation options
This repo is focused to showcase different options to implement "chat with your private documents" scenario using RAG patterns with Java, Azure OpenAI and Semantic Kernel. Below you can find the list of available implementations.
| Conversational Style | RAG Approach | Description | Java Open AI SDK | Java Semantic Kernel |
|---|---|---|---|---|
| One Shot Ask | PlainJavaAskApproach | Use Azure AI Search and Java OpenAI APIs. It first retrieves top documents from search and use them to build a prompt. Then, it uses OpenAI to generate an answer for the user question.Several search retrieval options are available: Text, Vector, Hybrid. When Hybrid and Vector are selected an additional call to OpenAI is required to generate embeddings vector for the question. | :white_check_mark: | :x: |
| Chat | PlainJavaChatApproach | Use Azure AI Search and Java OpenAI APIs. It first calls OpenAI to generate a search keyword for the chat history and then answer to the last chat question. Several search retrieval options are available: Text, Vector, Hybrid. When Hybrid and Vector are selected an additional call to OpenAI is required to generate embeddings vector for the chat extracted keywords. | :white_check_mark: | :x: |
| One Shot Ask | JavaSemanticKernelWithMemoryApproach | Use Java Semantic Kernel framework with built-in MemoryStore for embeddings similarity search. A semantic function RAG.AnswerQuestion is defined to build the prompt using Memory Store vector search results.A customized version of SK built-in CognitiveSearchMemoryStore is used to map index fields populated by the documents ingestion process. | :x: | This approach is currently disabled within the UI, memory feature will be available in the next java Semantic Kernel GA release |
| One Shot Ask | JavaSemanticKernelChainsApproach | Use Java Semantic Kernel framework with semantic and native functions chaining. It uses an imperative style for AI orchestration through semantic kernel functions chaining. InformationFinder.SearchFromQuestion native function and RAG.AnswerQuestion semantic function are called sequentially. Several search retrieval options are available: Text, Vector, Hybrid. | :x: | :white_check_mark: |
| Chat | JavaSemanticKernelWithMemoryApproach | Use Java Semantic Kernel framework with built-in MemoryStore for embeddings similarity search. A semantic function RAG.AnswerConversation is defined to build the prompt using Memory Store vector search results. A customized version of SK built-in CognitiveSearchMemoryStore is used to map index fields populated by the documents ingestion process. | :x: | :x: This approach is currently disabled within the UI, memory feature will be available in the next java Semantic Kernel GA release |
| Chat | JavaSemanticKernelChainsApproach | Use Java Semantic Kernel framework with semantic and native functions chaining. It uses an imperative style for AI orchestration through semantic kernel functions chaining. InformationFinder.SearchFromConversation native function and RAG.AnswerConversation semantic function are called sequentially. Several search retrieval options are available: Text, Vector, Hybrid. | :x: | :white_check_mark: |
Production deployment on Azure Application Landing Zones
This sample is designed to get you started quickly and let you experiment with Java intelligent Apps RAG architectures on Azure. For production deployment, you can use the Azure Application Landing Zones (LZA) to deploy the solution maintaining best practices for security, monitoring, networking and operational excellence.
Check the chat-with-your-data-lza-app-accelerator to see how you can deploy this solution on App Service LZA or Azure Container Apps LZA.
