(fix) add linting check to ban creating `AsyncHTTPHandler` during LLM calling

Add linting check to ban creating AsyncHTTPHandler(), this can increase latency by ~900ms
migrated all AsyncHTTPHandler() to use get_async_httpx_client
ensure all httpx clients stored in cache are cached for 1 hour (like how we do for OpenAI/Azure), this prevents ConnectionError bugs

Relevant issues

Type

🆕 New Feature 🐛 Bug Fix 🧹 Refactoring 📖 Documentation 🚄 Infrastructure ✅ Test

Changes

[REQUIRED] Testing - Attach a screenshot of any new tests passing locall

If UI changes, send a screenshot/GIF of working UI fixes

Description by Korbit AI

What change is being made?

Add a linting check to ensure that AsyncHTTPHandler is not instantiated directly during LLM calls, and replace its usage with get_async_httpx_client.

Why are these changes being made?

Direct instantiation of AsyncHTTPHandler can lead to inconsistencies and issues in the handling of HTTP requests. Using get_async_httpx_client ensures a more standardized and safer approach by consolidating client creation logic. This change introduces a new test, ensure_async_clients_test.py, to automatically catch any violations in the codebase, thereby enforcing best practices and code maintainability.

Is this description stale? Ask me to generate a new description by commenting /korbit-generate-pr-description

BerriAI / litellm