Wingman AI allows you to use your voice to talk to various AI providers and LLMs, process your conversations, and ultimately trigger actions such as pressing buttons or reading answers. Our Wingmen are like characters and your interface to this world, and you can easily control their behavior and characteristics, even if you're not a developer.
1.5.0 Showreel:
Release trailer:
In-depth tutorial:
AI is complex and it scares people. It's also not just ChatGPT. We want to make it as easy as possible for you to get started. That's what Wingman AI is all about. It's a framework that allows you to build your own Wingmen and use them in your games and programs.
The idea is simple, but the possibilities are endless. For example, you could:
Since version 2.0, Wingman AI Core acts as a "backend" API (using FastAPI and Pydantic) with the following features:
We (Team ShipBit) offer an additional client with a neat GUI that you can use to configure everything in Wingman AI Core.
No, it is not! We presented an early prototype of Wingman AI in Star Citizen on YouTube, which caused a lot of excitement and interest in the community. Star Citizen is a great game, we love it and it has a lot of interesting use-cases for Wingmen but it's not the only game we play and not the core of our interest. We're also not affiliated with CIG or Star Citizen in any way.
The video that started it all:
Wingman AI is an external, universal tool that you can run alongside any game or program. As such, it does not currently interact directly with Star Citizen or any other game, other than its ability to trigger system-wide keystrokes, which of course can have an effect on the game. However, if you find a way to interact with a game, either through an API or by reading the game's memory, you could - in theory - use it to directly trigger in-game actions or feed your models with live data. This is not the focus of Wingman AI, though.
The project is intended for two different groups of users:
If you're a developer, you can just clone the repository and start building your own Wingmen. We try to keep the codebase as open and hackable as possible, with lots of hooks and extension points. The base classes you'll need are well documented, and we're happy to help you get started. We also provide a development guide to help you witht the setup. Wingman AI Core is currently 100% written in Python.
If you're not a developer, you can start with pre-built Wingmen from us or from the community and adapt them to your needs. Since version 2, we offer an eay-to-use client for Windows that you can use to cofigure every single detail of your Wingmen. It also handles multiple configurations and offers system-wide settings like audio device selection.
Wingman AI Core is free but the AI providers you'll be using might not be. We know that this is a big concern for many people, so we are offering "Wingman Pro" which is a subscription-based service with a flat fee for all the AI providers you need (and additional GUI features). This way, you won't have to worry about intransparent "pay-per-use" costs.
Check out the pricing and features here: Wingman AI Pro
Wingman AI also supports local providers that you have to setup on your own but can then use and connect with our client for free:
You can also use your own API key to use the following services:
Our Wingmen use OpenAI's APIs and they charge by usage. That means: You don't pay a flat subscription fee, but rather for each call you make to their APIs. You can find more information about the APIs and their pricing on the OpenAI website. You will need to create your API key:
You don't have to use ElevenLabs as TTS provider, but their voices are great. You can also clone your own with less than 5 minutes of sample audio, e.g. your friend, an actor or a recording of an NPC in your game.
They have a free tier with a limited number of characters generated per month so you can try it out first. You can find more information on their pricing page.
Signing up is very similar to OpenAI: Create your account, set up your payment method, and create an API key.
Microsoft Edge TTS is actually free and you don't need an API key to use it. However, it's not as "good" as the others in terms of quality. Their voices are split by language, so the same voice can't speak different languages - you have to choose a new voice for the new language instead. Wingman does this for you, but it's still "Windows TTS" and not as good as the other providers.
Wingman AI exposes the base_url
property that the OpenAI Python client uses. So if you have a plug-in replacement for OpenAI's client, you can easily connect it to Wingman AI Core. You can also write your own custom Wingman that uses your local LLM.
Integrating specific LLMs oder models is currently not on our (ShipBit) priority list as explained here and we do not offer live support for it. Check out or Discord server if you're interested in local LLMs - there is a vibrant community discussing and testing different solutions and if we ever find one that satisfies our requirements, we might consider supporting it officially.
Wingman AI.exe
.
Wingman AI Core.exe
in the backgroundIf that doesn't work for some reason, try starting Wingman AI Core.exe
manually and check the terminal or your logs directory for errors.
If you're a developer, you can also run from source. This way you can preview our latest changes on the develop
branch and debug the code.
Wingman runs well on MacOS. While we don't offer a precompiled package for it, you can run it from source. Note that the TTS provider XVASynth is Windows-only and therefore not supported on MacOS.
Linux is not officially supported but some of our community members were able to run it anyways. Check out their documentation.
Our default Wingmen serve as examples and starting points for your own Wingmen, and you can easily reconfigure them using the client. You can also add your own Wingmen.
Our first two default Wingmen are using OpenAI's APIs. The basic process is as follows:
Talking to a Wingman is like chatting with ChatGPT. This means that you can customize their behavior by giving them a context
(or system
) prompt as starting point for your conversation. You can also just tell them how to behave and they will remember that during your conversation. ATC and Computer use very different prompts, so they behave very differently.
The magic happens when you configure commands or key bindings. GPT will then try to match your request with the configured commands and execute them for you. It will automatically choose the best matching command based only on its name, so make sure you give it a good one (e.g. RequestLandingPermission
).
More information about the API can be found in the OpenAI API documentation.
StarHead is where it gets really interesting. This Wingman is tailored to Star Citizen and uses the StarHead API to enrich your gaming experience with external data. It is a showcase of how to build specialized Wingmen for specific use-cases and scenarios. Simply ask StarHead for the best trade route, and it will prompt you for your ship, location, and budget. It will then call the StarHead API and read the result back to you.
Like all of our OpenAI Wingmen, it will remember the conversation history and you can ask follow-up questions. For example, you can ask what the starting point of the route, or what the next stop is. You can also ask for the best trade route from a different location or with a different ship.
StarHead is a community project that aims to provide a platform for Star Citizen players to share their knowledge and experience. At the moment it is mainly focused on the trading aspect of Star Citizen. With a huge database of trade items, shop inventories and prices, it allows you to find the best trade routes and make the most profit. A large community of players is constantly working to keep the data up to date.
For updates and more information, visit the StarHead website or follow @KNEBEL on
Yes, you can! You can edit all the configs in your %APP_DATA%\ShipBit\WingmanAI\[version]
directory.
The YAML configs are very indentation-sensitive, so please be careful. We recommend using VSCode with the YAML extension to edit them.
There is no hot reloading, so you have to restart Wingman AI Core after you made changes to the configs.
Use these naming conventions to create different configurations for different games or scenarios:
_[name]
(underscore): marks the default configuration that is launched on start, e.g. _Star Citizen
.wingmen
by adding [name].yaml
files. Do not use special characters.
.[name].yaml
(dot): marks the Wingman as "hidden" and skips it in the UI and on start, e.g. .Computer.yaml
.[name].png
(image): Sets an avatar for the Wingman in the client, e.g. StarHead.png
.There are a couple of other files and directories in the config directory that you can use to configure Wingman AI.
defaults.yaml
- contains the default settings for all Wingmen. This is merged with the settings of the individual Wingmen at runtime. Specific wingman settings always override the defaults. Once a wingman is saved using the client, it contains all the settings it needs to run and will no longer fallback to the defaults.settings.yaml
- contains user settings like the selected audio input and output devicessecrets.yaml
- contains the API keys for different providers.Access secrets in code by using secret_keeper.py
. You can access everything else with config_manager.py
.
Wingman supports all languages that OpenAI (or your configured AI provider) supports. Setting this up in Wingman is really easy:
Find the context
setting for the Wingman you want to change.
Now add a simple sentence to the context
prompt: Always answer in the language I'm using to talk to you.
or something like Always answer in Portuguese.
The cool thing is that you can now trigger commands in the language of your choice without changing/translating the name
of the commands - the AI will do that for you.
Also note that depending on your TTS provider, you might have to pick a voice that can actually speak your desired language or you'll end up with something really funny (like an American voice trying to speak German).
Are you ready to build your own Wingman or implement new features to the framework?
Please follow our guides to setup your dev environment:
If you want to read some code first and understand how it all works, we recommend you start here (in this order):
http://127.0.0.1:8000/docs
- The OpenAPI (ex: Swagger) specwingman_core.py
- most of the public API endpoints that Wingman AI exposes%APP_DATA%\ShipBit\WingmanAI\[version]
to get an idea of what's configurable.Wingman.py
- the base class for all WingmenOpenAIWingman.py
- derived from Wingman, using all the providersTower.py
- the factory that creates WingmenIf you're planning to develop a major feature or new integration, please contact us on Discord first and let us know what you're up to. We'll be happy to help you get started and make sure your work isn't wasted because we're already working on something similar.
Thank you so much for your support. We really appreciate it!
Wingman makes use of other Open Source projects internally (without modifying them in any way). We would like to thank their creators for their great work and contributions to the Open Source community.
This list will inevitably remain incomplete. If you miss your name here, please let us know in Discord or via Patreon.
To our greatest Patreon supporters we say: o7
Commanders!
Ira Robinson aka Serene/BlindDadDoes, Zenith, DiVille, [Hiwada], Hades aka Architeutes, Raziel317, CptToastey, NeyMR AKA EagleOne (Capt.Epic), a Bit Brutal, AlexeiX, Dragon Aura, Perry-x-Rhodan, DoublarThackery, SilentCid, Bytebool, Exaust A.K.A Nikoyevitch, Tycoon3000, N.T.G, Jolan97, Greywolfe, Dayel Ostraco aka BlakeSlate, Nielsjuh01, Manasy, Sierra-Noble, Simon says aka Asgard, JillyTheSnail, [Admiral-Chaos aka Darth-Igi], The Don, Tristan Import Error, Munkey the pirate, Norman Pham aka meMidgety, meenie, Tilawan, Mr. Moo42, Geekdomo, Jenpai, Blitz, Aaron Sadler, SleeperActual, parawho, HypeMunkey, Huniken, SuperTruck, [NozDog], Skipster [Skipster Actual], Fredek, Ruls-23, Dexonist, Captain Manga