Okik is a powerful command-line interface (CLI) tool designed to simplify the process of running various services using different frameworks on any cloud platform. With Okik, users can effortlessly deploy and manage their services directly on any cloud infrastructure without the complexity of handling their own infrastructure. This tool bridges the gap between development and deployment, allowing teams to focus on creating innovative applications rather than getting bogged down in infrastructure management.
Key features of Okik include:
Okik streamlines the development and deployment process, allowing developers to focus on building great applications rather than managing infrastructure. It abstracts away the complexities of cloud deployment, making it accessible to developers of all skill levels. Whether you're working on a small project or a large-scale enterprise application, Okik provides the flexibility and ease-of-use to meet your needs.
By leveraging Okik, teams can significantly reduce the time and resources spent on infrastructure management, leading to faster development cycles and more frequent releases. It also promotes best practices in deployment and scaling, helping to ensure that your applications are robust and performant.
For more information, including detailed documentation, tutorials, and community support, visit okik.ai.
Using pip
pip install okikThis is the simplest method to install Okik. It will automatically handle all dependencies and install the latest stable version of Okik.
Or To install Okik from source, follow these steps:
git clone https://github.com/okikorg/okik.gitcd okikpip install .This method is useful if you want to contribute to Okik's development or need the latest features that haven't been released in the stable version yet.
To run Okik, simply execute the following command in your terminal:
okik
██████ ██ ██ ██ ██ ██
██ ██ ██ ██ ██ ██ ██
██ ██ █████ ██ █████
██ ██ ██ ██ ██ ██ ██
██████ ██ ██ ██ ██ ██
Simplify. Deploy. Scale.
Type 'okik --help' for more commands.This command launches the Okik CLI, providing you with an overview of available commands and options.
okik initThis command sets up a new Okik project in your current directory. It creates necessary configuration files and project structure, preparing your environment for Okik-managed deployments.
Write this in your main.py file:
from okik.endpoints import service, endpoint, app
import asyncio
from typing import Any
from sentence_transformers import SentenceTransformer
import sentence_transformers
from torch.nn.functional import cosine_similarity as cosine
import torch
import random
# your service configuration
@service(
replicas=1,
resources={"accelerator": {"type": "A40", "device": "cuda", "count": 1, "memory": 4}},
backend="okik" # <- provisioning backend is okik
)
class Embedder:
def __init__(self):
self.model = SentenceTransformer("paraphrase-MiniLM-L6-v2", cache_folder=".okik/cache")
@endpoint()
def embed(self, sentence: str):
logits = self.model.encode(sentence)
return logits
@endpoint()
def similarity(self, sentence1: str, sentence2: str):
logits1 = self.model.encode(sentence1, convert_to_tensor=True)
logits2 = self.model.encode(sentence2, convert_to_tensor=True)
return cosine(logits1.unsqueeze(0), logits2.unsqueeze(0))
@endpoint()
def version(self):
return sentence_transformers.__version__
@endpoint(stream=True)
async def stream_data(self) -> Any:
async def data_generator():
for i in range(10):
yield f"data: {i}\n"
await asyncio.sleep(1)
return data_generator()
# Mock LLM Service Example
@service(replicas=1)
class MockLLM:
def __init__(self):
pass
@endpoint(stream=True) # <- streaming response enabled for use cases like chatbot
async def stream_random_words(self, prompt: str = "Hello"):
async def word_generator():
words = ["hello", "world", "fastapi", "stream", "test", "random", "words", "python", "async", "response"]
for _ in range(10):
word = random.choice(words)
yield f"{word}\n"
await asyncio.sleep(0.4)
return word_generator()
This example demonstrates how to create services and endpoints using Okik. The @service decorator defines a service with specific configuration, while @endpoint decorators define individual API endpoints within the service. The example includes both synchronous and asynchronous endpoints, as well as streaming capabilities.
# run the okik routes to check all available routes
okik routes# output should be similar to this
main.py Application Routes
├── <HOST>/health/
│ └── /health | GET
├── <HOST>/embedder/
│ ├── /embedder/embed | POST
│ ├── /embedder/similarity | POST
│ ├── /embedder/stream_data | POST
│ └── /embedder/version | POST
└── <HOST>/mockllm/
└── /mockllm/stream_random_words | POSTThis command displays all the routes defined in your application, helping you verify that your endpoints are correctly set up.
# run the okik run to start the server in production mode
okik server
# or run in dev mode
okik server --dev --reload
#or
okik server -d -rThese commands start your Okik server. The --dev and --reload flags are useful during development as they enable auto-reloading when code changes are detected.
curl -X POST http://0.0.0.0:3000/embedder/version
# or if you like to use httpie then
http POST 0.0.0.0:3000/embedder/version
# or test the stream endpoint
curl -X POST http://0.0.0.0:3000/mockllm/stream_random_words -d '{"prompt": "Hello"}'
# or if you like to use httpie then
http POST 0.0.0.0:3000/mockllm/stream_random_words prompt="hello" --streamThese commands demonstrate how to test your Okik endpoints using curl or httpie. They show both regular POST requests and how to handle streaming responses.
okik build -a "your_awesome_app" -t latestThis command builds your application, preparing it for deployment. The -a flag specifies the application name, and -t sets the tag for the build.
okik deployThis command deploys your built application to the configured cloud provider.
# similar to kubectl commands, infact you can use kubectl commands as well
okik get deployments # for deployments
okik get services # for servicesThese commands allow you to monitor your deployed applications and services, providing information about their status and configuration.
okik delete deployment "your_awesome_app"This command removes a deployed application from your cloud environment.
Okik is currently in development so expect sharp edges and bugs. Feel free to contribute to the project by submitting a pull request. Your feedback and contributions are valuable in improving and stabilizing Okik for the wider development community.