Prioritization implementation updates

[stuartleeks]

Purpose

Start aligning prioritization implementation variables and headers (and combine e2e tests)

• Rename vars in simple to match token-counting
• Add remaining-tokens/request headers to token-counting response
• Add low-priority rate-limit reason to token-counting response
• Add sliding window average to rate-limit token usage chart
• Update token-counting implementation to check x-priority header
• Move query trimming for terminal links into QueryProcessor
• Extend common prioritization e2e test
  • Add cycle2 load shape to common prioritization e2e test based on token-counting test
  • convert max_token users to chat completions (embeddings endpoint doesn't support max_tokens parameter)
  • Add missing queries
• merge prioritization-simple and prioritization-token-counting folders
• rename token counting API from AOAIAPI-PrioritizationTokenCounting to AOAIAPI-Prioritization-TokenCounting

Does this introduce a breaking change?

[ ] Yes
[ ] No

Pull Request Type

What kind of change does this Pull Request introduce?

[ ] Bugfix
[ ] Feature
[ ] Code style update (formatting, local variables)
[ ] Refactoring (no functional changes, no api changes)
[ ] Documentation content changes
[ ] Other... Please describe:

How to Test

• Get the code

git clone [repo-address]
cd [repo-name]
git checkout [branch-name]
npm install

• Test the code

What to Check

Verify that the following are valid

• ...

Other Information