search

langchain-ai / langchain-google

MIT License
98 stars 115 forks source link

`VertexAI` shall allow transparent passing of api_transport = "rest" #105

Closed cregouby closed 4 months ago

cregouby commented 5 months ago

Description

Since version 1.43.0, google-vertexai-aiplatform has added transport override to enable the use of REST instead of GRPC (6ab4084).
This is a key feature for enterprise where security setup prevent gRPC communications, and thus for large scale adoption.

Current behavior

from langchain_google_vertexai import VertexAI
llm = VertexAI(model_name="text-bison@002", temperature=0, location="europe-west4", api_transport="rest")

leads to a gRPC error message (expected here as gRPC is filtered by organisation) 

E0329 15:25:35.266053000 10916507648 ssl_transport_security.cc:1519]   Handshake failed with fatal error SSL_ERROR_SSL: error:1000007d:SSL routines:OPENSSL_internal:CERTIFICATE_VERIFY_FAILED.
I0329 15:25:35.266485000 10916507648 subchannel.cc:744]                subchannel 0x105f0b7d0 {address=ipv4:1.2.4.5:3128, args={grpc.client_channel_factory=0x16c067ac0, grpc.default_authority=europe-west4-aiplatform.googleapis.com:443, grpc.dns_enable_srv_queries=1, grpc.http2_scheme=https, grpc.http_connect_headers=Proxy-Authorization:Basic d-----------------j, grpc.http_connect_server=europe-west4-aiplatform.googleapis.com:443, grpc.internal.channel_credentials=0x16c068c90, grpc.internal.event_engine=0x105f09d30, grpc.internal.security_connector=0x105f0afe0, grpc.internal.subchannel_pool=0x28a36f530, grpc.internal.transport=(nil), grpc.max_receive_message_length=-1, grpc.max_send_message_length=-1, grpc.primary_user_agent=grpc-python/1.62.0, grpc.resource_quota=0x28a343f50, grpc.server_uri=dns:///europe-west4-aiplatform.googleapis.com:443}}: connect failed (UNKNOWN:Ssl handshake failed: SSL_ERROR_SSL: error:1000007d:SSL routines:OPENSSL_internal:CERTIFICATE_VERIFY_FAILED {created_time:"2024-03-29T15:25:35.266102+01:00", tsi_error:"TSI_PROTOCOL_FAILURE", tsi_code:10}), backing off for -4543 ms

Expected behavior

api_transport="rest"is transparently passed as transport to the vertexai.language_models Models

With any error message (if any) being related to HTTP, not gRPC.

lkuligin commented 5 months ago

thanks for raising a feature request!

@cregouby would you like to contribute yourself maybe?

cregouby commented 5 months ago

I would be glad to propose a contribution, even if I'm not sure I'll be technically able to. Any hint on where to start on ? Thanks !

lkuligin commented 5 months ago

I'd suppose you'll need to add additional param to _VertexAICommon (with the default value) and pass it here: https://github.com/langchain-ai/langchain-google/blob/62cdf5285d35d7700bf5d52f5cfdbfcf84d8f1d7/libs/vertexai/langchain_google_vertexai/_base.py#L159

cregouby commented 4 months ago

Hello,

It seems that https://github.com/langchain-ai/langchain-google/pull/183 fix the issue.