Documentation - API Reference - Changelog - Bug reports - Discord
⚠️ Cortex is currently in Development: Expect breaking changes and bugs!
Cortex is an OpenAI-compatible AI engine that developers can use to build LLM apps. It is packaged with a Docker-inspired command-line interface and client libraries. It can be used as a standalone server or imported as a library.
Cortex currently supports 3 inference engines:
Ensure that your system meets the following requirements to run Cortex:
Visit Quickstart to get started.
# Install using NPM
npm i -g cortexso
# Install using Brew
brew tap janhq/cortexso
brew install cortexso
# Run model
cortex run llama3
# To uninstall globally using NPM
npm uninstall -g cortexso
# Install using Brew
brew tap janhq/cortexso
brew install cortexso
# Run model
cortex run llama3
# To uninstall using Brew
brew uninstall cortexso
brew untap janhq/cortexso
You can also install Cortex using the Cortex Installer available on GitHub Releases.
To run Cortex as an API server:
cortex serve
# Output
# Started server at http://localhost:1337
# Swagger UI available at http://localhost:1337/api
You can now access the Cortex API server at http://localhost:1337
,
and the Swagger UI at http://localhost:1337/api
.
To install Cortex from the source, follow the steps below:
cortex-js
folder.npx nest build
command.js
executable:chmod +x '[path-to]/cortex/cortex-js/dist/src/command.js'
npm link
The following CLI commands are currently available. See CLI Reference Docs for more information.
serve Providing API endpoint for Cortex backend
chat Send a chat request to a model
init|setup Init settings and download cortex's dependencies
ps Show running models and their status
kill Kill running cortex processes
pull|download Download a model. Working with HuggingFace model id.
run [options] EXPERIMENTAL: Shortcut to start a model and chat
models Subcommands for managing models
models list List all available models.
models pull Download a specified model.
models remove Delete a specified model.
models get Retrieve the configuration of a specified model.
models start Start a specified model.
models stop Stop a specified model.
models update Update the configuration of a specified model.
benchmark Benchmark and analyze the performance of a specific AI model using your system.
presets Show all the available model presets within Cortex.
telemetry Retrieve telemetry logs for monitoring and analysis.