Concurrency limiter and scheduler

[tanveergill]

Description of change

Checklist

• [x] Tested in playground or other setup
• [ ] Screenshot (Grafana) from playground added to PR for 15+ minute run
• [ ] Documentation is changed or added
• [ ] Tests and/or benchmarks are included
• [ ] Breaking changes

Summary by CodeRabbit

• New Features

  • Introducing new flow control capabilities with the addition of FlowEnd RPC.
  • Enhanced decision-making with ConcurrencyLimiterInfo and ConcurrencySchedulerInfo.
  • Expanded HTTP checks including expect_end and check_response fields for better control.

• Improvements

  • Updated concurrency and rate limiting configurations for more precise control.
  • Improved policy definitions for concurrency limiting and scheduling.

• Documentation

  • Added new documentation for concurrency limiting and scheduling blueprints.
  • Provided comprehensive guides for configuring and utilizing new flow control features.

• Monitoring

  • Introduced new Grafana panels for monitoring concurrency limiter and scheduler metrics.

• Refactor & Code Quality

  • Refactored metrics processing with enhanced error handling and token return functionalities.
  • Streamlined concurrency scheduler and rate limiter implementations for better performance and maintainability.

• Bug Fixes

  • Addressed issues in policy actuators to ensure correct decision-making and scheduling.

[coderabbitai[bot]]

Walkthrough

The updates primarily introduce new features and modifications related to flow control in a distributed system. This includes the implementation of concurrency limiters and schedulers, enhancements to the request handling process, and the introduction of new RPCs. Additionally, there are updates to configuration files, policy definitions, and dashboard components to support these features, as well as changes to token counter and rate limiter functionalities to improve system performance and observability.

Changes

File Path	Change Summary
api/aperture/flowcontrol/check/v1/..., api/aperture/flowcontrol/checkhttp/v1/...	Added FlowEnd RPC, expect_end fields, ConcurrencyLimiterInfo, ConcurrencySchedulerInfo, and related messages. Included expect_end and check_response fields in CheckHTTPRequest.
api/aperture/policy/language/v1/..., api/aperture/policy/private/v1/..., api/aperture/policy/sync/v1/...	Updated and added new concurrency-related messages and structures.
blueprints/concurrency-limiting/base/..., blueprints/concurrency-scheduling/base/...	Provided new documentation, configurations, and policy definitions for concurrency limiting and scheduling.
dashboards/grafana/dashboards/summary/...	Added new Grafana panels for concurrency limiters and schedulers.
pkg/dmap-funcs/..., pkg/metrics/..., pkg/otelcollector/metricsprocessor/..., pkg/policies/flowcontrol/actuators/...	Introduced and updated packages for token counters, metrics, processors, and actuators.
docs/content/reference/blueprints/concurrency-scheduling/...	Created documentation for concurrency scheduling policies.

🐇✨ In the land of code where the data streams flow,
New changes have come, with a bright, shiny glow.
Concurrency's tamed, with limiters so wise,
A rabbit's delight, under digital skies.
🌟🌙

---

Tips

### Chat with CodeRabbit Bot (`@coderabbitai`) - You can directly reply to a review comment made by CodeRabbit. Example: - `I pushed a fix in commit ``.` - You can tag CodeRabbit on specific lines of code or entire files in the PR by tagging `@coderabbitai` in a comment. Examples: - `@coderabbitai generate unit tests for this file` - `@coderabbitai modularize this function` - You can tag `@coderabbitai` in a PR comment and ask questions about the PR and the codebase. Use quoted replies to pass the context for follow-up questions. Examples: - `@coderabbitai gather interesting statistics about this repository and render them in a table.` - `@coderabbitai show all the `console.log` statements in this repository.` - `@coderabbitai generate unit tests for the `src/utils.ts` file.` ### CodeRabbit Commands (invoked as PR comments) - `@coderabbitai pause` to pause the reviews on a PR. - `@coderabbitai resume` to resume the paused reviews. - `@coderabbitai review` to trigger a review. This is useful when automatic reviews are disabled for the repository. - `@coderabbitai resolve` resolve all the CodeRabbit review comments. - `@coderabbitai help` to get help. Additionally, you can add `@coderabbitai ignore` anywhere in the PR description to prevent this PR from being reviewed. ### CodeRabbit Configration File (`.coderabbit.yaml`) - You can programmatically configure CodeRabbit by adding a `.coderabbit.yaml` file to the root of your repository. - The JSON schema for the configuration file is available [here](https://coderabbit.ai/integrations/coderabbit-overrides.v2.json). - If your editor has YAML language server enabled, you can add the path at the top of this file to enable auto-completion and validation: `# yaml-language-server: $schema=https://coderabbit.ai/integrations/coderabbit-overrides.v2.json` ### CodeRabbit Discord Community Join our [Discord Community](https://discord.com/invite/GsXnASn26c) to get help, request features, and share feedback.