Open Smitraj007 opened 11 months ago
Our LLM model is on endpoint :
os.environ["OPENAI_API_BASE"] = "my_endpoint_ip" os.environ["OPENAI_API_KEY"] = "empty" embedding = OpenAIEmbeddings(model="text-embedding-ada-002") local_llm = OpenAI(model="gpt-3.5-turbo")
So while performing below retrieval task we are not able to fetch llm model from endpoint. qa_chain = RetrievalQA.from_chain_type(llm=local_llm, chain_type="stuff", retriever=retriever, return_source_documents=True)
ConnectionResetError Traceback (most recent call last) File ~/SageMaker/AmazonSageMaker-IAG/virtual_environment/lib/python3.10/site-packages/urllib3/connectionpool.py:714, in HTTPConnectionPool.urlopen(self, method, url, body, headers, retries, redirect, assert_same_host, timeout, pool_timeout, release_conn, chunked, body_pos, **response_kw) 713 # Make the request on the httplib connection object. --> 714 httplib_response = self._make_request( 715 conn, 716 method, 717 url, 718 timeout=timeout_obj, 719 body=body, 720 headers=headers, 721 chunked=chunked, 722 ) ConnectionError: ('Connection aborted.', ConnectionResetError(104, 'Connection reset by peer'))
same error
You can refer to this. https://github.com/lm-sys/FastChat/issues/2303#issuecomment-2151829969
Our LLM model is on endpoint :
from langchain.indexes import VectorstoreIndexCreator
os.environ["OPENAI_API_BASE"] = "my_endpoint_ip" os.environ["OPENAI_API_KEY"] = "empty" embedding = OpenAIEmbeddings(model="text-embedding-ada-002") local_llm = OpenAI(model="gpt-3.5-turbo")
So while performing below retrieval task we are not able to fetch llm model from endpoint. qa_chain = RetrievalQA.from_chain_type(llm=local_llm, chain_type="stuff", retriever=retriever, return_source_documents=True)
ERROR : ConnectionResetError
ConnectionResetError Traceback (most recent call last) File ~/SageMaker/AmazonSageMaker-IAG/virtual_environment/lib/python3.10/site-packages/urllib3/connectionpool.py:714, in HTTPConnectionPool.urlopen(self, method, url, body, headers, retries, redirect, assert_same_host, timeout, pool_timeout, release_conn, chunked, body_pos, **response_kw) 713 # Make the request on the httplib connection object. --> 714 httplib_response = self._make_request( 715 conn, 716 method, 717 url, 718 timeout=timeout_obj, 719 body=body, 720 headers=headers, 721 chunked=chunked, 722 ) ConnectionError: ('Connection aborted.', ConnectionResetError(104, 'Connection reset by peer'))