Might just be my complete n00biness with LLM prompt mechanics, but trying to follow examples, I'm running into trouble trying to set up TheBloke/Wizard-Vicuna-13B-Uncensored-GPTQ to use Wikipedia as a langchain tool. Basically this issue. Here's my test code (demo.py):
import os
from langchain import OpenAI, LLMChain
from langchain.agents import Tool, AgentExecutor
from langchain.utilities import WikipediaAPIWrapper
from ogbujipt.config import *
from ogbujipt.model_styles.wizard_vicuna import wv_output_parser, wv_prompt, wv_agent_factory
API_HOST = 'http://192.168.1.10' # Points to my local machine running Ooba + OpenAI
os.environ['OPENAI_API_BASE'] = f'{API_HOST}:{DEFAULT_API_PORT}/v1'
# Set up the API connector
llm = OpenAI(temperature=0.1)
# Set up Wikipedia as an agent tool
wp_wrapped = WikipediaAPIWrapper()
tools = [
Tool(
name="Wikipedia",
func=wp_wrapped.run,
description="The collaboratively curated encyclopedia, Wikipedia, a great source of facts."
# Maybe I shouldn't tell the bot the truth about Wikipedia 😂
# description="The collaboratively curated encyclopedia, Wikipedia, a great source of facts, and occasionally, nonsense."
)
]
tool_names = [tool.name for tool in tools]
output_parser = wv_output_parser()
# Everything but the tools is already prepped for the WV prompt
prompt = wv_prompt(tools=tools)
llm_chain = LLMChain(llm=llm, prompt=prompt)
agent = wv_agent_factory(
llm_chain=llm_chain,
output_parser=output_parser,
allowed_tools=tool_names)
agent_executor = AgentExecutor.from_agent_and_tools(
agent=agent, tools=tools, verbose=True
)
agent_executor.run("Who won this year's NBA Western Conference finals?") # Mile hi! 😎
Weird stuff happens besides the "is not a valid tool" bits. I get output from Ooba like below. It seems to think this year is 2021?
Ooba logs while handling the above request
```
text-generation-webui | USER:Answer the following questions as best you can. You have access to the following tools:
text-generation-webui |
text-generation-webui | Wikipedia: The collaboratively curated encyclopedia, Wikipedia, a great source of facts.
text-generation-webui |
text-generation-webui | Use the following format:
text-generation-webui |
text-generation-webui | Question: the input question you must answer
text-generation-webui | Thought: you should always think about what to do
text-generation-webui | Action: the action to take, should be one of Wikipedia
text-generation-webui | Action Input: the input to the action
text-generation-webui | Observation: the result of the action
text-generation-webui | ... (this Thought/Action/Action Input/Observation can repeat N times)
text-generation-webui | Thought: I now know the final answer
text-generation-webui | Final Answer: the final answer to the original input or the final conclusion to your thoughts
text-generation-webui |
text-generation-webui |
text-generation-webui | Begin!
text-generation-webui |
text-generation-webui | Question: Who won this year's NBA Western Conference finals?
text-generation-webui | ASSISTANT: \Thought:
text-generation-webui | Thought: Let me check on Wikipedia for information on NBA Western Conference finals.
text-generation-webui | Action: Searching for "NBA Western Conference Finals" in Wikipedia.
text-generation-webui | Action Input: https://en.wikipedia.org/wiki/2021_NBA_Playoffs#Western_Conference_Finals
text-generation-webui | Observation: Searching for "NBA Western Conference Finals" in Wikipedia. is not a valid tool, try another one.
text-generation-webui | Thought: \Thought:
text-generation-webui | Action: Checking if there are any other sources that provide information on NBA Western Conference finals.
text-generation-webui | Action Input: Googling "who won nba western conference finals".
text-generation-webui | Observation: Checking if there are any other sources that provide information on NBA Western Conference finals. is not a valid tool, try another one.
text-generation-webui | Thought: \Thought:
text-generation-webui | Action: Asking someone who knows about basketball and NBA.
text-generation-webui | Action Input: Asking my friend who is an avid fan of NBA.
text-generation-webui | Observation: Asking someone who knows about basketball and NBA. is not a valid tool, try another one.
text-generation-webui | Thought: \Thought:
text-generation-webui | Action: Reading news articles related to NBA Western Conference finals.
text-generation-webui | Action Input: Googling "nba western conference finals news".
text-generation-webui | Observation: Reading news articles related to NBA Western Conference finals. is not a valid tool, try another one.
text-generation-webui | Thought: \Thought:
text-generation-webui | Action: Watching sports channels like ESPN or TNT which cover NBA games extensively.
text-generation-webui | Action Input: Turning on ESPN channel.
text-generation-webui | Observation: Watching sports channels like ESPN or TNT which cover NBA games extensively. is not a valid tool, try another one.
text-generation-webui | Thought:
text-generation-webui | --------------------
text-generation-webui |
text-generation-webui | Output generated in 32.86 seconds (1.03 tokens/s, 34 tokens, context 562, seed 208203058)
```
Might just be my complete n00biness with LLM prompt mechanics, but trying to follow examples, I'm running into trouble trying to set up TheBloke/Wizard-Vicuna-13B-Uncensored-GPTQ to use Wikipedia as a langchain tool. Basically this issue. Here's my test code (
demo.py
):Weird stuff happens besides the "is not a valid tool" bits. I get output from Ooba like below. It seems to think this year is 2021?
Ooba logs while handling the above request
``` text-generation-webui | USER:Answer the following questions as best you can. You have access to the following tools: text-generation-webui | text-generation-webui | Wikipedia: The collaboratively curated encyclopedia, Wikipedia, a great source of facts. text-generation-webui | text-generation-webui | Use the following format: text-generation-webui | text-generation-webui | Question: the input question you must answer text-generation-webui | Thought: you should always think about what to do text-generation-webui | Action: the action to take, should be one of Wikipedia text-generation-webui | Action Input: the input to the action text-generation-webui | Observation: the result of the action text-generation-webui | ... (this Thought/Action/Action Input/Observation can repeat N times) text-generation-webui | Thought: I now know the final answer text-generation-webui | Final Answer: the final answer to the original input or the final conclusion to your thoughts text-generation-webui | text-generation-webui | text-generation-webui | Begin! text-generation-webui | text-generation-webui | Question: Who won this year's NBA Western Conference finals? text-generation-webui | ASSISTANT: \Thought: text-generation-webui | Thought: Let me check on Wikipedia for information on NBA Western Conference finals. text-generation-webui | Action: Searching for "NBA Western Conference Finals" in Wikipedia. text-generation-webui | Action Input: https://en.wikipedia.org/wiki/2021_NBA_Playoffs#Western_Conference_Finals text-generation-webui | Observation: Searching for "NBA Western Conference Finals" in Wikipedia. is not a valid tool, try another one. text-generation-webui | Thought: \Thought: text-generation-webui | Action: Checking if there are any other sources that provide information on NBA Western Conference finals. text-generation-webui | Action Input: Googling "who won nba western conference finals". text-generation-webui | Observation: Checking if there are any other sources that provide information on NBA Western Conference finals. is not a valid tool, try another one. text-generation-webui | Thought: \Thought: text-generation-webui | Action: Asking someone who knows about basketball and NBA. text-generation-webui | Action Input: Asking my friend who is an avid fan of NBA. text-generation-webui | Observation: Asking someone who knows about basketball and NBA. is not a valid tool, try another one. text-generation-webui | Thought: \Thought: text-generation-webui | Action: Reading news articles related to NBA Western Conference finals. text-generation-webui | Action Input: Googling "nba western conference finals news". text-generation-webui | Observation: Reading news articles related to NBA Western Conference finals. is not a valid tool, try another one. text-generation-webui | Thought: \Thought: text-generation-webui | Action: Watching sports channels like ESPN or TNT which cover NBA games extensively. text-generation-webui | Action Input: Turning on ESPN channel. text-generation-webui | Observation: Watching sports channels like ESPN or TNT which cover NBA games extensively. is not a valid tool, try another one. text-generation-webui | Thought: text-generation-webui | -------------------- text-generation-webui | text-generation-webui | Output generated in 32.86 seconds (1.03 tokens/s, 34 tokens, context 562, seed 208203058) ```