Closed karansohi closed 10 months ago
The recent updates across various documentation files focus on the implementation of gRPC deadlines within API calls. These changes highlight the importance of setting a maximum duration for requests to remain in a queue for prioritization purposes, ensuring that requests are either processed or discarded in a timely manner. The updates also refine the decision-making process related to request handling and telemetry.
File Path | Change Summary |
---|---|
.../api-quota-management.md |
Expanded logic for API limit handling with gRPC deadline in startFlow call. |
.../guides/openai.md |
Added instructions for setting gRPC deadline in startFlow call. |
.../introduction.md |
Removed bold formatting from "Caching". |
.../sdk/*/manual.md |
Emphasized setting higher gRPC deadline for request prioritization and clarified request handling. |
🐇✨ In the land of code, where the data streams flow, A rabbit hopped in, with deadlines in tow. "Set your timers," it squeaked with delight, "For requests to queue, then take flight!" 🚀 🐇✨
Description of change
Checklist
Summary by CodeRabbit
shouldRun
/should_run
andend
calls in request processing and telemetry data collection.