VSCode copilot initial run skips code after function diff

[obriensystems]

The following run is ok

michaelobrien@mbp7 rag % python app.py
USER_AGENT environment variable not set, consider setting it to identify your requests.
docs content size: 43131

      LLM Powered Autonomous Agents

Date: June 23, 2023  |  Estimated Reading Time: 31 min  |  Author: Lilian Weng

Building agents with LLM (large language model) as its core controller is a cool concept. Several proof-of-concepts demos, such as AutoGPT, GPT-Engineer and BabyAGI, serve as inspiring examples. The potentiality of LLM extends beyond generating well-written copies, stories, essays and programs; it can be framed as a powerful general problem solver.
Agent System Overview#
In
splits: 66
page_content: 969
metadata: 1
INFO:chromadb.telemetry.product.posthog:Anonymized telemetry enabled. See                     https://docs.trychroma.com/telemetry for more information.
INFO:httpx:HTTP Request: POST https://api.openai.com/v1/embeddings "HTTP/1.1 200 OK"
retriever: tags=['Chroma', 'OpenAIEmbeddings'] vectorstore=<langchain_chroma.vectorstores.Chroma object at 0x137954d30> search_kwargs={'k': 6}
INFO:httpx:HTTP Request: POST https://api.openai.com/v1/embeddings "HTTP/1.1 200 OK"
vectorstore retrieved: 6
vectorstore retrieved content: Tree of Thoughts (Yao et al. 2023) extends CoT by exploring multiple reasoning possibilities at each step. It first decomposes the problem into multiple thought steps and generates multiple thoughts per step, creating a tree structure. The search process can be BFS (breadth-first search) or DFS (depth-first search) with each state evaluated by a classifier (via a prompt) or majority vote.
Task decomposition can be done (1) by LLM with simple prompting like "Steps for XYZ.\n1.", "What are the subgoals for achieving XYZ?", (2) by using task-specific instructions; e.g. "Write a story outline." for writing a novel, or (3) with human inputs.
/Users/michaelobrien/Library/Python/3.9/lib/python/site-packages/langsmith/client.py:5431: LangChainBetaWarning: The function `loads` is in beta. It is actively being worked on, so the API may change.
  prompt = loads(json.dumps(prompt_object.manifest))
example_messages: You are an assistant for question-answering tasks. Use the following pieces of retrieved context to answer the question. If you don't know the answer, just say that you don't know. Use three sentences maximum and keep the answer concise.
Question: filler question 
Context: filler context 
Answer:
INFO:httpx:HTTP Request: POST https://api.openai.com/v1/embeddings "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai.com/v1/chat/completions "HTTP/1.1 200 OK"
rag_chain retrieved: 375
rag_chain retrieved content: Task decomposition is the process of breaking down complex tasks into smaller, more manageable steps. This is often achieved through techniques like Chain of Thought (CoT), which encourages step-by-step reasoning, or Tree of Thoughts, which explores multiple reasoning possibilities at each step. It can be facilitated by prompts, task-specific instructions, or human inputs.

For the code

# from https://python.langchain.com/v0.2/docs/tutorials/rag/
import getpass
import os
import bs4

from langchain_openai import ChatOpenAI
from langchain import hub
from langchain_chroma import Chroma
from langchain_community.document_loaders import WebBaseLoader
from langchain_core.output_parsers import StrOutputParser
from langchain_core.runnables import RunnablePassthrough
from langchain_openai import OpenAIEmbeddings
from langchain_text_splitters import RecursiveCharacterTextSplitter
import logging

logging.basicConfig(level=logging.INFO)

#getpass.getpass()

os.environ["LANGCHAIN_TRACING_V2"] = "true"
#os.environ["OPENAI_API_KEY"] = "sk-p..."
os.environ["OPENAI_API_KEY"] = "sk-...gA"

#os.environ["LANGCHAIN_API_KEY"] = "lsv..."
os.environ["LANGCHAIN_API_KEY"] = "lsv...69"

#getpass.getpass()

llm = ChatOpenAI(model="gpt-4o-mini")

# Load, chunk and index the contents of the blog.
loader = WebBaseLoader(
    web_paths=("https://lilianweng.github.io/posts/2023-06-23-agent/",),
    bs_kwargs=dict(
        parse_only=bs4.SoupStrainer(
            class_=("post-content", "post-title", "post-header")
        )
    ),
)
docs = loader.load()

print(f"docs content size: %s" % (len(docs[0].page_content)))
print(docs[0].page_content[:500])

text_splitter = RecursiveCharacterTextSplitter(
   chunk_size=1000, chunk_overlap=200) #  add_start_index=True
splits = text_splitter.split_documents(docs)
print(f"splits: %s" % (len(splits)))
print(f"page_content: %s" % (len(splits[0].page_content)))
print(f"metadata: %s" % (len(splits[10].metadata)))

vectorstore = Chroma.from_documents(documents=splits, embedding=OpenAIEmbeddings())

  # Retrieve and generate using the relevant snippets of the blog.
#retriever = vectorstore.as_retriever() # default simularity, 4
retriever = vectorstore.as_retriever(search_type="similarity", 
                                     search_kwargs={"k": 6})

print (f"retriever: %s" % (retriever))
retrieved_docs = retriever.invoke("What are the approaches to Task Decomposition?")
print(f"vectorstore retrieved: %s" % (len(retrieved_docs)))
print(f"vectorstore retrieved content: %s" % (retrieved_docs[0].page_content))

def format_docs(docs):
    return "\n\n".join(doc.page_content for doc in docs)

prompt = hub.pull("rlm/rag-prompt")

example_messages = prompt.invoke(
    {"context": "filler context", "question": "filler question"}
).to_messages()
#example_messages
print(f"example_messages: %s" % (example_messages[0].content))

rag_chain = (
  {"context": retriever | format_docs, "question": RunnablePassthrough()}
   | prompt
   | llm
   | StrOutputParser()
  )

#for chunk in rag_chain.stream("What is Task Decomposition?"):
#    print(chunk, end="", flush=True)

retrieved = rag_chain.invoke("What is Task Decomposition?")
print(f"rag_chain retrieved: %s" % (len(retrieved)))
print(f"rag_chain retrieved content: %s" % (retrieved))#[0].page_content))

github clean up "go through my code on the right in app.py again"

import os
import bs4
from langchain_openai import ChatOpenAI, OpenAIEmbeddings
from langchain_chroma import Chroma
from langchain_community.document_loaders import WebBaseLoader
from langchain_text_splitters import RecursiveCharacterTextSplitter
import logging

# Configure logging
logging.basicConfig(level=logging.INFO)

def initialize_llm():
    return ChatOpenAI(model="gpt-4o-mini")

def set_environment_variables():
    os.environ["LANGCHAIN_TRACING_V2"] = "true"
    # os.environ["OPENAI_API_KEY"] = "your_openai_api_key"
    # os.environ["LANGCHAIN_API_KEY"] = "your_langchain_api_key"

def load_documents():
    loader = WebBaseLoader(
        web_paths=("https://lilianweng.github.io/posts/2023-06-23-agent/",),
        bs_kwargs=dict(
            parse_only=bs4.SoupStrainer(
                class_=("post-content", "post-title", "post-header")
            )
        ),
    )
    return loader.load()

def split_documents(docs):
    text_splitter = RecursiveCharacterTextSplitter(
        chunk_size=1000, chunk_overlap=200
    )
    return text_splitter.split_documents(docs)

def create_vectorstore(splits):
    return Chroma.from_documents(documents=splits, embedding=OpenAIEmbeddings())

def retrieve_documents(vectorstore, query, k=6):
    retriever = vectorstore.as_retriever(search_type="similarity", search_kwargs={"k": k})
    return retriever.invoke(query)

def main():
    set_environment_variables()
    llm = initialize_llm()

    try:
        docs = load_documents()
        logging.info(f"Docs content size: {len(docs[0].page_content)}")
        logging.info(docs[0].page_content[:500])

        splits = split_documents(docs)
        logging.info(f"Splits: {len(splits)}")
        logging.info(f"Page content size: {len(splits[0].page_content)}")
        logging.info(f"Metadata size: {len(splits[10].metadata)}")

        vectorstore = create_vectorstore(splits)
        retrieved_docs = retrieve_documents(vectorstore, "What are the approaches to Task Decomposition?")
        logging.info(f"Vectorstore retrieved: {len(retrieved_docs)}")
    except Exception as e:
        logging.error(f"An error occurred: {e}")

if __name__ == "__main__":
    main()

This issue is everything after the following function definition is skipped - even though that is where the rag prompt is run against the LLM

def format_docs(docs):
    return "\n\n".join(doc.page_content for doc in docs)

[obriensystems]

It seems like a cr/lf or special character truncated output - as I commented and uncommented the two print lines and we fully refactor until nearer the end now - previously we stopped before the 2nd print

H

        logging.info(f"Vectorstore retrieved: {len(retrieved_docs)}")

now we stop before

retrieved = rag_chain.invoke("What is Task Decomposition?")
print(f"rag_chain retrieved: %s" % (len(retrieved)))
print(f"rag_chain retrieved content: %s" % (retrieved))#[0].page_content))

def initialize_llm():
    return ChatOpenAI(model="gpt-4o-mini")

def load_documents():
    loader = WebBaseLoader(
        web_paths=("https://lilianweng.github.io/posts/2023-06-23-agent/",),
        bs_kwargs=dict(
            parse_only=bs4.SoupStrainer(
                class_=("post-content", "post-title", "post-header")
            )
        ),
    )
    return loader.load()

def split_documents(docs):
    text_splitter = RecursiveCharacterTextSplitter(
        chunk_size=1000, chunk_overlap=200
    )
    return text_splitter.split_documents(docs)

def create_vectorstore(splits):
    return Chroma.from_documents(documents=splits, embedding=OpenAIEmbeddings())

def retrieve_documents(vectorstore, query, k=6):
    retriever = vectorstore.as_retriever(search_type="similarity", search_kwargs={"k": k})
    return retriever.invoke(query)

def format_docs(docs):
    return "\n\n".join(doc.page_content for doc in docs)

def main():
    set_environment_variables()
    llm = initialize_llm()

    try:
        docs = load_documents()
        logging.info(f"Docs content size: {len(docs[0].page_content)}")
        logging.info(docs[0].page_content[:500])

        splits = split_documents(docs)
        logging.info(f"Splits: {len(splits)}")
        logging.info(f"Page content size: {len(splits[0].page_content)}")
        logging.info(f"Metadata size: {len(splits[10].metadata)}")

        vectorstore = create_vectorstore(splits)
        retrieved_docs = retrieve_documents(vectorstore, "What are the approaches to Task Decomposition?")
        logging.info(f"Vectorstore retrieved: {len(retrieved_docs)}")

        prompt = hub.pull("rlm/rag-prompt")
        example_messages = prompt.invoke(
            {"context": "filler context", "question": "filler question"}
        ).to_messages()
        logging.info(f"Example messages: {example_messages[0].content}")

        rag_chain = (
            {"context": retriever | format_docs, "question": RunnablePassthrough()}
            | prompt
            | llm
            | StrOutputParser()
        )

        # Uncomment to use the RAG chain
        # for chunk in rag_chain.stream("What is Task Decomposition?"):
        #     logging.info(chunk)

    except Exception as e:
        logging.error(f"An error occurred: {e}")

if __name__ == "__main__":
    main()

[obriensystems]

removed the last comment section "for chunk" - better

import logging

# Assuming the necessary imports and initializations are done above this snippet

def main():
    # ... other code ...

    try:
        rag_chain = (
            {"context": retriever | format_docs, "question": RunnablePassthrough()}
            | prompt
            | llm
            | StrOutputParser()
        )

        retrieved = rag_chain.invoke("What is Task Decomposition?")
        logging.info(f"rag_chain retrieved: {len(retrieved)}")
        logging.info(f"rag_chain retrieved content: {retrieved}")

    except Exception as e:
        logging.error(f"An error occurred: {e}")

if __name__ == "__main__":
    main()

[obriensystems]

however retriever is not global anymore - it needs to be taken out of the function scope

 rag_chain = (
            {"context": retriever | format_docs, "question": RunnablePassthrough()}