dosubot[bot]
commented
7 months ago 🤖
Hello @shikanime! I'm Dosu, your friendly bot helper. I'm here to assist with bugs, answer your queries, and guide you on becoming a contributor. While we wait for a human maintainer to join us, feel free to ask me anything. Remember, no question is too small!
Thank you for your feature request. It seems like you're asking for a new class, similar to SageMakerEndpoint, that provides an interface for interacting with Vertex AI Endpoints. This class would allow users to easily connect to and interact with Vertex AI Endpoints and would be versatile enough to handle custom models with unique input and output formats.
Based on the current structure of LangChain, there are already classes for interacting with Vertex AI Endpoints, namely the VertexAI and VertexAIModelGarden classes. These classes provide methods for generating text, streaming text, and counting tokens, among other things. You can find more details about these classes in the test_vertexai.py file.
However, if these classes do not meet your needs, it would be helpful to have more specific details about the functionality you're looking for. For example, what methods would you like the new class to have? How should these methods behave? What kind of input and output formats should the class support?
Regarding the SageMakerEndpoint class, it is a part of the LLM (Language Model) module and is used to interact with a deployed Sagemaker model. It has several attributes and methods, including a client attribute for the Boto3 client for Sagemaker runtime, an endpoint_name attribute for the name of the endpoint from the deployed Sagemaker model, and a _call method for calling out to the Sagemaker inference endpoint. You can find more details about this class in the sagemaker_endpoint.py file.
I hope this information is helpful. If you have any further questions or if there's anything else you'd like to discuss, please let me know.
Best regards, Dosu
Sources
- .scripts/community_split/libs/community/tests/integration_tests/llms/test_vertexai.py
- libs/langchain/langchain/llms/sagemaker_endpoint.py
This response is meant to be useful and save you time. It is not meant to be a precise solution, but rather a starting point for your own research.
Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant.
If you want to continue the conversation, start your reply with @dosu-bot.
Feature request
Add first-class support for Vertex AI Endpoints in Langchain. This would involve providing a similar interface to the existing SageMakerEndpoint class, allowing users to easily connect to and interact with Vertex AI Endpoints.
Motivation
Although VertexAIModelGarden already exist, there may be instances where users require custom models with unique input and output formats. To address this need, a more versatile class could be developed, upon which VertexAIModelGarden could be built. This would allow for seamless integration of custom models without compromising the functionality of the existing Model Garden class.
Your contribution
The implementation taking inspiration from SageMakerEndpoint if pertinent.