There are many 1000s of Gradio apps on Hugging Face Spaces. This library puts them at the tips of your LLM's fingers 🦾
Specifically, gradio-tools
is a Python library for converting Gradio apps into tools that can be leveraged by a large language model (LLM)-based agent to complete its task. For example, an LLM could use a Gradio tool to transcribe a voice recording it finds online and then summarize it for you. Or it could use a different Gradio tool to apply OCR to a document on your Google Drive and then answer questions about it.
Currently supported libraries for agents are:
gradio-tools
comes with a set of pre-built tools you can leverage immediately! These include:
We welcome more contributions!
Simply import the desired tools from gradio_tools
(or create your own!) and pass to initialize_agent
from LangChain.
In this example, we use some pre-built tools to generate images, caption them, and create a video!
Read the How It Works section to learn how to create your own tools! We welcome any new tools to the library!
from gradio_tools import (StableDiffusionTool, ImageCaptioningTool, StableDiffusionPromptGeneratorTool,
TextToVideoTool)
from langchain.agents import initialize_agent
from langchain.llms import OpenAI
from langchain.memory import ConversationBufferMemory
llm = OpenAI(temperature=0)
memory = ConversationBufferMemory(memory_key="chat_history")
tools = [StableDiffusionTool().langchain, ImageCaptioningTool().langchain,
StableDiffusionPromptGeneratorTool().langchain, TextToVideoTool().langchain]
agent = initialize_agent(tools, llm, memory=memory, agent="conversational-react-description", verbose=True)
output = agent.run(input=("Please create a photo of a dog riding a skateboard "
"but improve my prompt prior to using an image generator."
"Please caption the generated image and create a video for it using the improved prompt."))
See the /examples
directory for more complete code examples.
The core abstraction is the GradioTool
, which lets you define a new tool for your LLM as long as you implement a standard interface:
class GradioTool(BaseTool):
def __init__(self, name: str, description: str, src: str) -> None:
@abstractmethod
def create_job(self, query: str) -> Job:
pass
@abstractmethod
def postprocess(self, output: Tuple[Any] | Any) -> str:
pass
The requirements are:
freddyaboulton/calculator
, of the Gradio application. Based on this value, gradio_tools
will create a gradio client instance to query the upstream application via API. Be sure to click the link and learn more about the gradio client library if you are not familiar with it.submit
function of the client. More info on creating jobs here_block_input(self, gr)
and _block_output(self, gr)
methods of the tool. The gr
variable is the gradio module (the result of import gradio as gr
). It will be
automatically imported by the GradiTool
parent class and passed to the _block_input
and _block_output
methods.And that's it!
A LangChain agent is a Large Language Model (LLM) that takes user input and reports an output based on using one of many tools at its disposal.
Gradio is the defacto standard tool for building Machine Learning Web Applications and sharing them with the world - all with just python! 🐍