search

MicrosoftDocs / azure-docs

Open source documentation of Microsoft Azure
https://docs.microsoft.com/azure
Creative Commons Attribution 4.0 International
10.31k stars 21.5k forks source link

Clarify "Limit" is RPM. or TPM in this page #110767

Closed tsg-ash-kanagat closed 8 months ago

tsg-ash-kanagat commented 1 year ago

Whenever the term "Limit" is used, indicate if it's Rate per Minute (RPM) of API calls or Tokens per Minute (TPM)

Document Details

⚠ Do not edit this section. It is required for learn.microsoft.com ➟ GitHub issue linking.

mrbullwinkle commented 1 year ago

@tsg-ash-kanagat we go in depth into how the rate limits work here: https://learn.microsoft.com/azure/cognitive-services/openai/how-to/quota#understanding-rate-limits

When we are referring to best practices regarding rate limits in the quotas & limits article these practices like employing retry logic in your application apply to both RPM & TPM.

tsg-ash-kanagat commented 1 year ago

To clarify- there are references made on document pages that only say “Limit”, and no qualification on whether it is referring to RPM or TPM.

I would expect the text in document pages to say “Limit (RPM)” or “Limit (TPM)”.

From: Michael @.> Date: Monday, June 12, 2023 at 9:23 AM To: MicrosoftDocs/azure-docs @.> Cc: Kanagat, Ash @.>, Mention @.> Subject: Re: [MicrosoftDocs/azure-docs] Clarify "Limit" is RPM. or TPM in this page (Issue #110767)

@tsg-ash-kanagathttps://urldefense.com/v3/__https:/github.com/tsg-ash-kanagat__;!!AbgBjg!2ApfYbWRlgIoCXaCRvFy-L20B35-x1jJjcZ69h5JiU3p-hHJlJkLnhEzKalngqSUojx_xUyWotnnbod_0G8KC7ia$ we go in depth into how the rate limits work here: https://learn.microsoft.com/azure/cognitive-services/openai/how-to/quota#understanding-rate-limitshttps://urldefense.com/v3/__https:/learn.microsoft.com/azure/cognitive-services/openai/how-to/quota*understanding-rate-limits__;Iw!!AbgBjg!2ApfYbWRlgIoCXaCRvFy-L20B35-x1jJjcZ69h5JiU3p-hHJlJkLnhEzKalngqSUojx_xUyWotnnbod_0B3q1f_v$

When we are referring to best practices regarding rate limits in the quotas & limits article these practices like employing retry logic in your application apply to both RPM & TPM.

— Reply to this email directly, view it on GitHubhttps://urldefense.com/v3/__https:/github.com/MicrosoftDocs/azure-docs/issues/110767*issuecomment-1587337643__;Iw!!AbgBjg!2ApfYbWRlgIoCXaCRvFy-L20B35-x1jJjcZ69h5JiU3p-hHJlJkLnhEzKalngqSUojx_xUyWotnnbod_0AmGlTaU$, or unsubscribehttps://urldefense.com/v3/__https:/github.com/notifications/unsubscribe-auth/ASMNEEZZ7RWN54CTHX6CLGLXK4J4JANCNFSM6AAAAAAZDLDCYQ__;!!AbgBjg!2ApfYbWRlgIoCXaCRvFy-L20B35-x1jJjcZ69h5JiU3p-hHJlJkLnhEzKalngqSUojx_xUyWotnnbod_0Ccf5PeZ$. You are receiving this because you were mentioned.Message ID: @.***>

This e-mail, including any attachments, contains confidential information of Bain & Company, Inc. ("Bain") and/or its clients. It may be read, copied and used only by the intended recipient. Any use by a person other than its intended recipient, or by the recipient but for purposes other than the intended purpose, is strictly prohibited. If you received this e-mail in error, please contact the sender and then destroy this e-mail. Opinions, conclusions and other information in this message that do not relate to the official business of Bain shall be understood to be neither given nor endorsed by Bain. Any personal information sent over e-mail to Bain will be processed in accordance with our Privacy Policy (https://www.bain.com/privacy).

Naveenommi-MSFT commented 1 year ago

@tsg-ash-kanagat Thanks for your feedback! We will investigate and update as appropriate.

RamanathanChinnappan-MSFT commented 1 year ago

@tsg-ash-kanagat

I've delegated this to @mrbullwinkle, a content author, to review and share their valuable insights.

mrbullwinkle commented 8 months ago

Thanks for this feedback, the docs have been updated to help clarify TPM/RPM.

mrbullwinkle commented 8 months ago

please-close