Generative AI Information

[santokalayil]

All possible Generative AI information can be put here

flowchart LR;
prompts --> LLM

Some useful links:

• https://learn.microsoft.com/en-us/dotnet/ai/conceptual/how-genai-and-llms-work

[santokalayil]

Prompt Engineering

• Process of creating the best, accurate and specific prompts (use templates if possible) to instruct the generative AI models to output the best possible responses

Prompt Templates

Common prompt structures

1. Instruction based prompts (Zero shot prompting)
2. Role based prompts
3. System prompts
4. Few shot learning prompts (Few short prompting): Provide few examples and expected outputs for them and ask the question directly to LLM
5. Chain-of-Thought prompting

[santokalayil]

Projects Done

• Based on Company and report year, renaming and categorizing company reports pdfs (huge list- not named properly)
• RAG Implementation of company pdf reports to answer questions (helpful for domain experts). This cut down the time of manually opening, reading and researching on the company data
• Categorizing the SEDOLs to different sectors using Description found in database using RAG. Those which has very less probability were given to domain experts to manually look into
• Classification problem using Few short learning prompts directly using GPT3.5 Turbo

[santokalayil]

LLMS

[santokalayil]

Some Concepts:

Context Window (tokens) : ability to feed data into the LLMs (how much we can feed external data)

[santokalayil]

LLM models

• Anthropic's Claude 3, 2.1
• Google's Gemini 1.5, ..
• OpenAI's GPT-4
• Mistral's Mixtral
• Mistral's Mistral
• LLaMMA 2, 3
• Phi-2, 3
• Falcon

Embedding Models

• Cohere Multilingual Embedding model - closed source (best in class)
• OpenAI Embedding model - cheaper compared to. Uses Cosine similarity often. Normalized to length 1 (so that it can be computed just with dot product). Euclidean distance as well. have context window of around 8000 tokens
• Sentence Transformers - open source. slow but robust. converts each sentense into vector embeddings. supports wide variety of models.

  from sentence_transformers import SentenceTransformer
  em = SentenceTranformer("paraphrase-MiniLM-L6-v2")
  embedding = em.encode("my sentence")

• FastEmbed: Lightweight, Faster than sentense transformers. supports GPU. Limited models supported

  from fastembed import TextEmbedding
  em = TextEmbedding()
  embeddings: List = list(em.embed(["doc1.....", "doc2....."]))

[santokalayil]

Examples of Vector Databases

1. Chroma DB - LangChain (Python and JavScript), LlamaIndex, support available
2. Pinecone
3. Faiss - By Facebook

[santokalayil]

RAG vs CRAG (Corrective) vs RAFT (RAG + finetuning) + PEFT (parameter efficient fine tuning), LORA, QLORA

RAG:

https://python.langchain.com/v0.2/docs/tutorials/rag/

[santokalayil]

https://github.com/santokalayil/NLP/blob/main/notebooks/notebook_05_spacy.ipynb

[santokalayil]

Technology Stack for Generative AI

Learn Scikit LLM