search

thinkgradient / sustainability-with-openai-km

3 stars 1 forks source link

readme

Sunstainability demo using OpenAI

Knowledge Mining Solution Accelerator - Code Samples | Microsoft Docs

This folder contains a Knowledge Mining Solution Accelerator solution that leverages OpenAI model. Specific steps need to be applied and relevant configuration according to reader's needs. CognitiveSearch.UI < Search.UI folder is a .NET Core MVC Web app used a Template UI for querying a search index.

This is an end-to-end solution containing Cognitive Services and the focus of this README file is to prepare the reader into leveraging easily this accelerator.

Technical requirements

Prerequisites

1. Create a resource group

In your Azure subscription, create a resource group (please make sure it's unique using your initials or initial+last_name e.g. fi-openai-workshop or fismali-openai-workshop).
In the same resource group, deploy an Azure Key Vault instance.

How to create a resource group 1.1 Create from Cloud Shell 1.2 Create from Portal 1.3 Create from Azure PowerShell
How to create a key vault 1.1 Create from Cloud Shell 1.2 Create from Portal 1.3 Create from Azure Power Shell 
Important: Add the key vault name that you just created in kvname in the main.parameters.json file

2 Clone the following repo: Sustainability with Openai - Knowledge Mining

How to to clone a repo

3. Run the Bicep deployment

Open the paramenters.json file in an editor of your choice and add the relevant values for the following two keys:

Replace the resource group name you have created in Step 1 in the below command:

az deployment group create --resource-group <resource_group_name> --template-file main.bicep --parameters main.parameters.json

After you execute the deployment command, for the first time you will get the following error which for now you can ignore.

Re-run the Bicep deployment command again one more time.

If you hit the following error, then you have to deploy the Video Indexer function code. Please see step 3 below. Configure tenant setting as per Step 3 and then deploy the Video Index (start-video-indexing) function in Step 4 and re-run the Bicep command again.

3. Configure VS Code for your tenant

4c. Once you deploy the VideIndexer function carry out the following steps:

4f. Add the following two role assignments to Video Indexer service

6. Run the Postman collections request

  1. Import the Collections json and Environments json file into Postman

In Postman click on the following button Import in the Colleaction tab (on the left side) <img src="https://user-images.githubusercontent.com/106242539/207662467-1ce3f72c-c47e-4fea-a2dc-4d6822e4dd70.png" width="800" height="270">

Then choose the tab File and from your local folder select sustainability-with-openai-km < Postman-Collections < Search Components.postman_collection

Repeat the step above but for the environment variables and select sustainability-with-openai-km < Postman-Collections < Search Env Params.postman_environment <img src="https://user-images.githubusercontent.com/106242539/207662632-9b53e0c5-2d29-4ce3-b567-460f91db45d3.png" width="800" height="270">

  1. Change the environment variables to the corresponding value by looking at the service name and service keys from the resources in the relevant resource group.

Click on the Environments tab on the left and then on the tab that you have just created, Airlift2022 Search Env Params. You need to replace both the INITIAL_VALUE and CURRENT_VALUE with the ones that have been created on your resource group on Azure Portal.

Replace all of the following variables and keys with the ones from the Azure Portal EXCEPT:

7. Running the Collections requests one by one in this order:

Return to Postman and run the Collections requests as instructed below, one-by-one by clicking the Send button. Execute them in this order: 1 --> 2 --> 3 --> 4 --> 9

These requests will create the required Search components, namely:

8. Search UI Functionality

9. Deploy using Docker

To run the docker commands:

Navigate to SearchUI folder and run:

  1. docker build -f Docker\Dockerfile --tag <IMAGE_NAME> . (Note: don't forget the dot '.' at the end of the command)

Also make sure that the above is the same value as the dockerName parameter value from the sustainability-with-openai-km\Bicep\main.parameters.json file. Once the docker image is built carry out the following commands to push it.

To push the docker image from your local docker server to the Container Registry

  1. az login --> to login to your azure account

  2. az account set -- subscription <SUBSCRIPTION_ID>

  3. az acr login --name <CONTAINER_REGISTRY>

  4. docker tag <IMAGE_NAME> <CONTAINER_REGISTRY>.azurecr.io/<IMAGE_NAME>

  5. docker push <CONTAINER_REGISTRY>.azurecr.io/<IMAGE_NAME>

Note: Wait about 15 minutes till everything has been pushed and check weblink to view the accelerator

9. Inspect the Search UI

From the Azure Portal find the App Service and navigate to its URL or click the Browse button. The URL has a .azurewebsites.net suffix.

10. Process PDF files

Open Storage Explorer and upload the PDF document in the data folder from the git cloned repo to the documents container. Refresh the Storage Explorer (click on the three ellipses at the top right and click Refresh). After a few seconds you should see a processed folder with all the individual pages of the original PDF file split into individual PDF files.

10. Process Video files

Open Storage Explorer and upload the Video file in the data folder from the git cloned repo to the video-knowledge-mining-drop container.

Appendix

Helpful commands

dir - to navigate between folder while in command prompt
docker container ls -a (list all containers)
docker images [OPTIONS] [REPOSITORY[:TAG]] (list docker images)
docker rm [OPTIONS] CONTAINER [CONTAINER...] (remove container)