🥷 FluxNinja Aperture
Aperture is a distributed load management platform designed for rate limiting,
caching, and prioritizing requests in cloud applications. Built upon a
foundation of distributed counters, observability, and a global control plane,
it provides a comprehensive suite of load management capabilities. These
capabilities enhance the reliability and performance of cloud applications,
while also optimizing cost and resource utilization.
![Unified Load Management](https://github.com/fluxninja/aperture/raw/main/docs/content/assets/img/unified-load-management-dark.svg#gh-dark-mode-only)
Integrating Aperture in your application through SDKs is a simple 3-step
process:
- Define labels: Define labels to identify users, entities, or features
within your application. For example, you can define labels to identify
individual users, features, or API endpoints.
Example
```typescript
// Tailor policies to get deeper insights into your workload with labels that
// capture business context.
const labels = {
// You can rate limit each user individually.
user: "jack",
// And have different rate limits for different tiers of users.
tier: "premium",
// You can also provide the tokens for each request.
// Tokens are flexible: LLM AI tokens in a prompt, complexity of a request,
// number of sub-actions, etc.
tokens: "200",
// When peak load exceeds external quotas or infrastructure capacity,
// requests can be throttled and prioritized.
priority: HIGH,
// Get deep insights into your workload. You can slice and dice performance
// metrics by any label.
workload: "/chat",
};
```
- Wrap your workload: Wrap your workload with
startFlow
and endFlow
calls to establish control points around specific features or code sections
inside your application. For example, you can wrap your API endpoints with
Aperture SDKs to limit the number of requests per user or feature.
Example
```typescript
// Wrap your workload with startFlow and endFlow calls, passing in the
// labels you defined earlier.
const flow = await apertureClient.startFlow("your_workload", {
labels: labels,
// Lookup result cache key to retrieve a cached result.
resultCacheKey: queryParams,
});
// If rate or quota limit is not exceeded, the workload is executed.
if (flow.shouldRun()) {
// Return a cached result or execute the workload.
const cachedResult = flow.resultCache();
const result = await yourWorkload(cachedResult);
flow.setResultCache({
value: result,
ttl: { seconds: 86400, nanos: 0 },
});
}
//
```
- Configure & monitor policies: Configure policies to control the rate,
concurrency, and priority of requests.
Policy YAML
```yaml
blueprint: rate-limiting/base
uri: github.com/fluxninja/aperture/blueprints@latest
policy:
policy_name: rate_limit
rate_limiter:
bucket_capacity: 60
fill_amount: 60
parameters:
interval: 3600s
limit_by_label_key: user
selectors:
- control_point: your_workload
label_matcher:
match_list:
- key: tier
operator: In
values:
- premium
```
![Rate Limiter Dashboard](https://github.com/fluxninja/aperture/raw/main/docs/content/get-started/assets/rate-limiter-dashboard-light.png#gh-light-mode-only)
In addition to language SDKs, Aperture also integrates with existing control
points such as API gateways, service meshes, and application middlewares.
⚙️ Load management capabilities
- ⏱️
Global Rate and Concurrency Limiting:
Safeguard APIs and features against excessive usage with Aperture's
high-performance, distributed rate limiter. Identify individual users or
entities by fine-grained labels. Create precise rate limiters controlling
burst-capacity and fill-rate tailored to business-specific labels. Limit per
user or global concurrency of in-flight requests. Refer to the
Rate Limiting and
Concurrency Limiting
guides for more details.
- 📊
API Quota Management:
Maintain compliance with external API quotas with a global token bucket and
smart request queuing. This feature regulates requests aimed at external
services, ensuring that the usage remains within prescribed rate limits and
avoids penalties or additional costs. Refer to the
API Quota Management
guide for more details.
- 🚦
Concurrency Control and Prioritization:
Safeguard against abrupt service overloads by limiting the number of
concurrent in-flight requests. Any requests beyond this limit are queued and
let in based on their priority as capacity becomes available. Refer to the
Concurrency Control and Prioritization
guide for more details.
- 🎯
Workload Prioritization:
Safeguard crucial user experience pathways and ensure prioritized access to
external APIs by strategically prioritizing workloads. With
weighted fair queuing,
Aperture aligns resource distribution with business value and urgency of
requests. Workload prioritization applies to API Quota Management and
Concurrency Control and Prioritization use cases.
- 💾 Caching: Boost application
performance and reduce costs by caching costly operations, preventing
duplicate requests to pay-per-use services, and easing the load on constrained
services. Refer to the Caching
guide for more details.
🏁 Getting Started
☁️ Aperture Cloud
[!NOTE]
FluxNinja has been acquired by CodeRabbit. New
sign-ups are temporarily disabled. Existing users can continue to use Aperture
Cloud by signing in to their accounts.
The easiest way to try Aperture is to
sign up for a free Aperture Cloud account.
Aperture Cloud is a fully managed service by FluxNinja. With Aperture Cloud,
there's no need to manage any infrastructure, and you can integrate your
application with Aperture using SDKs. For more information, refer to the
get started guide.
Prioritization Metrics for gpt-4
Performance Metrics for OpenAI Models
🎮 Local Kubernetes Playground
To try Aperture in a local Kubernetes environment, refer to
Playground docs.
📖 Learn More
🎥 Videos
👷 Contributing
Reporting bugs
helps us improve Aperture to be more reliable and user-friendly. Include all the
required information to reproduce and understand the bug you are reporting.
Follow helper questions in the bug report template to make it easier. If you see
a way to improve Aperture, use the
feature request
template to create an issue.
To contribute code, read the Contribution guide.