Open natebosch opened 5 months ago
Is there any reason why the package doesn't expose the ability to override the base URL and add custom headers? It's common practice to call LLM APIs through a proxy server to avoid hardcoding the API keys into the client apps. Currently, the only way to do this seems to be to provide a custom HTTP client that overrides the base URL and add the custom headers behind the scenes.
There is a lot of overlap between the services, so initially the vertex AI SDK will depend on this package for most data types and the base model implementation. We'll start out with
lib/src/
imports from the vertex AI SDK and extra caution when making changes here. When we are getting ready to publish as a stable version we should move to a better long term approach, or add mechanical checks to the current approach. In my opinion we should choose between making the extension points part of the stable public API, or moving the base implementation to a shared dependency and using the same customization handles to implement the vertex and google AI SDKs, or moving both implementations into a single package.