Jockey is a conversational video agent designed for complex video workflows. It combines the following technologies:
This allows Jockey to perform accurate video operations based on natural language instructions.
NOTE: Jockey is currently in alpha development. It may be unstable or behave unexpectedly. Use caution when implementing Jockey in production environments.
Use cases include but are not limited to the following:
Ensure the following prerequisites are met before installing and using Jockey.
Git: Any recent version.
git --version
command.Python: Version 3.11 or higher.
python3 --version
command.FFmpeg: Must be accessible in your system's PATH
environment variable.
PATH
environment variable.ffmpeg -version
command.Docker: Required for running the Jockey API server.
docker --version
command.Docker Compose V2: Required for running the Jockey API server.
docker compose version
command.
If you see a message similar to "docker: 'compose' is not a docker command," you may have v1. To update your Docker Compose version, see the Migrate to Compose V2 page of the official Docker documentation.Twelve Labs API Key:
LLM Provider API Key:
LangSmith API Key
jockey API server
, we need a LangGraph
API key. You can test LangGraph locally with the free developer plan.
LangGraph
(https://www.langchain.com/langgraph) and click the setting icon at the bottom left.API Keys
and click the Create API Key
at the top right.LANGSMITH_API_KEY
variable in your .env
file.This section guides you through the process of installing Jockey on your system. Please ensure all the prerequisites are met before proceeding with the installation. If you encounter any issues, please refer to the Troubleshooting page or reach out on the Multimodal Minds Discord server for assistance.
Open a terminal, navigate to the directory where you want to install Jockey, and enter the following command:
git clone https://github.com/twelvelabs-io/tl-jockey.git
cd tl-jockey && python3 -m venv venv
source venv/bin/activate
echo $VIRTUAL_ENV
The output should display the path to your virtual environment directory, as shown in the example below:
/Users/tl/jockey/tl-jockey/venv
This indicates that your virtual environment is activated. Your virtual environment is not activated if you see an empty line.
If this check indicates that your virtual environment is not activated, activate it using the source venv/bin/activate
command.
Install the required Python packages:
pip3 install -r requirements.txt
Jockey uses environment variables for configuration, and comes with an example.env
file to help you get started.
tl-jockey
directory, copy the example.env
file to a new file named .env
:.env
file in a text editor.Common variables
Variable | Description | Example |
---|---|---|
LANGSMITH_API_KEY |
Your Langgraph-sdk API key. | lsv2_... |
TWELVE_LABS_API_KEY |
Your Twelve Labs API key. | tlk_987654321 |
LLM_PROVIDER |
The LLM provider you wish to use. Possible values are AZURE and OPENAI . |
AZURE |
HOST_PUBLIC_DIR |
Directory for storing rendered videos | ./output |
HOST_VECTOR_DB_DIR |
Directory for vector database storage | ./vector_db |
LLM provider-specific variables
For Azure OpenAI:
Variable | Description | Example |
---|---|---|
AZURE_OPENAI_ENDPOINT |
Your Azure OpenAI endpoint URL | https://your-resource-name.openai.azure.com/ |
AZURE_OPENAI_API_KEY |
Your Azure OpenAI API key | 987654321 |
AZURE_OPENAI_API_VERSION |
The API version you're using | 2023-12-01-preview |
For OpenAI:
Variable | Description | Example |
---|---|---|
OPENAI_API_KEY |
Your OpenAI API key | 987654321 |
This section provides instructions on how to deploy and use Jockey. Note that Jockey supports the following deployment options:
This document covers the terminal-based deployment. If you're a developer looking to integrate Jockey into your application, see the Deploy and Use Jockey with the LangGraph API Server
The terminal deployment is ideal for quick testing, development work, and debugging. It provides immediate feedback and allows for easy interaction with Jockey.
Terminal Example Jockey Video Walkthrough
source venv/bin/activate
python3 -m jockey terminal
Use index 65f747a50db0463b8996bde2. I'm trying to create a funny video focusing on Gordon Ramsay. Can you find 3 clips of Gordon yelling at his chefs about scrambled eggs and then a final clip where Gordon bangs his head on a table. After you find all those clips, lets edit them together into one video.
Note that in some cases, such as summarizing videos or generating chapters and highlights, you must also provide a video ID. You can continue the conversation by providing new instructions or asking questions, as shown in the following example:
This is awesome but the last clip is too long. Lets shorten the last clip where Gordon hits his head on the table by making it start one second later. Then combine all the clips into a single video again.
Ctrl+C
keyboard shortcut.The terminal version of Jockey provides verbose output for debugging purposes:
To adjust the verbosity of the output, modify the parse_langchain_events_terminal()
function in jockey/util.py
.
Note that the tags for the individual components are set in app.py.
To integrate Jockey into your application, use an HTTP client library or the LangGraph Python SDK.
For a basic example of how to interact with Jockey programmatically, refer to the client.ipynb Jupyter notebook in the project repository. For more detailed information, see the LangGraph Examples page.