Closed JonAtDocuWare closed 1 year ago
I'm having the same issue. However, in my case, my deployment is also called text-embedding-ada-002
too but it's failing with the following error:
InvalidRequestError: Too many inputs for model None. The max number of inputs is 1. We hope to increase the number of inputs per request soon. Please contact us through an Azure support request at: https://go.microsoft.com/fwlink/?linkid=2213926 for further questions.
I printed the name of the model right before the call was done in the source code and it's correct, so no sure what's happening
Actually, by reading the error again, it seems like it's not related to the name of the model and azure just throttling.
So, in order to configure langchain embeddings properly, I have to replace this part with my desired chunk_size:
embeddings = embedding.embed_documents(texts, 1)
Of course, ideally this should be sent by parameter
I ran into the same issue with the chunk_size and Embeddings in Azure OpenAI Services and provided a fix. The easiest way is to initialize your OpenAIEmbeddings with chunk_size=1 - it works in other helper functions too when you can't pass the chunk_size e.g.
embeddings = OpenAIEmbeddings(chunk_size=1)
For the deployment name, it should work with names != text-embedding-ada-002
:
embeddings = OpenAIEmbeddings(engine=<your deployment name>)
@floleuerer wait, there's no AzureOpenAIEmbeddings
, is there? Were you referring to AzureOpenAI
instead?
Ah sorry my bad! it' s OpenAIEmbeddings
of course.
Thanks for confirming. Got excited to think they had a version for Azure! Although it doesn't make a big difference as it's just a config matter.
I just got to my laptop and tried. It fails with the following message:
pydantic.error_wrappers.ValidationError: 1 validation error for OpenAIEmbeddings
chunk_size
extra fields not permitted (type=value_error.extra)
Maybe I have the wrong version? I forked version 0.0.100 - could you confirm yours?
The fix should be in 0.0.106.
Gotcha! I'll try to fork the new version this week.
Is this issue fixed? Am on 0.0.107 version, but still getting below error with trying to use OpenAIEmbeddings with the azure openai service. Tried both FAISS as well as Pinecone vector stores. from_documents() fails with
openai.error.InvalidRequestError: Too many inputs for model None. The max number of inputs is 1. We hope to increase the number of inputs per request soon. Please contact us through an Azure support request at: https://go.microsoft.com/fwlink/?linkid=2213926 for further questions
I ran into the same issue with the chunk_size and Embeddings in Azure OpenAI Services and provided a fix. The easiest way is to initialize your OpenAIEmbeddings with chunk_size=1 - it works in other helper functions too when you can't pass the chunk_size e.g. .from_documents()...
embeddings = OpenAIEmbeddings(chunk_size=1)
For the deployment name, it should work with names !=
text-embedding-ada-002
:embeddings = OpenAIEmbeddings(engine=<your deployment name>)
@deb007 did you try @floleuerer solution?
I ran into the same issue with the chunk_size and Embeddings in Azure OpenAI Services and provided a fix. The easiest way is to initialize your OpenAIEmbeddings with chunk_size=1 - it works in other helper functions too when you can't pass the chunk_size e.g. .from_documents()...
embeddings = OpenAIEmbeddings(chunk_size=1)
For the deployment name, it should work with names !=
text-embedding-ada-002
:embeddings = OpenAIEmbeddings(engine=<your deployment name>)
@deb007 did you try @floleuerer solution?
Yes, I tried OpenAIEmbeddings(document_model_name="<deployment_name>", chunk_size=1)
- it gives error -
openai.error.InvalidRequestError: The API deployment for this resource does not exist. If you created the deployment within the last 5 minutes, please wait a moment and try again.
OpenAIEmbeddings(document_model_name="<deployment_name>")
gives error -
openai.error.InvalidRequestError: Too many inputs for model None. The max number of inputs is 1. We hope to increase the number of inputs per request soon. Please contact us through an Azure support request at: https://go.microsoft.com/fwlink/?linkid=2213926 for further questions.
OpenAIEmbeddings(engine="<deployment_name>")
gives below error:
extra fields not permitted (type=value_error.extra)
Actually, by reading the error again, it seems like it's not related to the name of the model and azure just throttling.
So, in order to configure langchain embeddings properly, I have to replace this part with my desired chunk_size:
embeddings = embedding.embed_documents(texts, 1)
Of course, ideally this should be sent by parameter
@deb007 this 👆 worked for me.
embeddings = OpenAIEmbeddings(chunk_size=1)
This also worked for me. Thanks!
However, what are the implications of setting chunk size to 1?
Unfortunately, I have tasted all the above methods and they have not worked, including deployment_name is set to text-davinci-003... Or change the version. I'm confused about openai.api_base and openai.api_type doesn't work. When I comment out the code, I find that there is no difference, and I still report an error as follows:
openai.error.InvalidRequestError: The API deployment for this resource does not exist. If you created the deployment within the last 5 minutes, please wait a moment and try again.
https://github.com/hwchase17/langchain/issues/1560#issue-1617563090 This solves my problem
embeddings = OpenAIEmbeddings(chunk_size=1)
This also worked for me. Thanks!
However, what are the implications of setting chunk size to 1?
I think latency (costs should be the same as it’s based on tokens and not API calls). If I’m not mistaken, you can add multiple texts in one call. I remember reading this somewhere but I couldn’t find it in their docs anymore. However, the error seems align with what I’m saying.
I'm on langchain-0.0.117 and as long as I use OpenAIEmbeddings()
without any parameters, it works smoothly with Azure OpenAI Service, but requires the model deployment to be named text-embedding-ada-002
.
@floleuerer @aajn88
Setting the chunk_size
explicity while initializing the OpenAIEmbeddings()
embeds worked for me.
Setting the deployment name explicitly did not help to overcome the max_input error.
langchain v=0.0.117
(latest)
@floleuerer @aajn88
Setting the
chunk_size
explicity while initializing theOpenAIEmbeddings()
embeds worked for me. Setting the deployment name explicitly did not help to overcome the max_input error.
langchain v=0.0.117
(latest)
Agreed, I was not able to get it to work with setting the model name manually.
I'm on langchain=0.0.119 but OpenAIEmbeddings()
throws an AuthenticationError: Incorrect API key provided... it seems that it tries to authenticate through the OpenAI API instead of the AzureOpenAI service, even when I configured the OPENAI_API_TYPE and OPENAI_API_BASE previously. Does anyone have the same problem?... tried with version 0.0.117 but the problem persists
I'm on langchain=0.0.119 but
OpenAIEmbeddings()
throws an AuthenticationError: Incorrect API key provided... it seems that it tries to authenticate through the OpenAI API instead of the AzureOpenAI service, even when I configured the OPENAI_API_TYPE and OPENAI_API_BASE previously. Does anyone have the same problem?... tried with version 0.0.117 but the problem persists
For some reason, the suggested implementation on the documentation (by setting environmental variables) does not work. However, a workaround is before openAIEmbedding
is ever called, import openai
package and set parameters manually
eg:
import openai
openai.api_base = "www.x.com/"
openai.api_type = 'azure'
openai.api_version = "2022-12-01"
# optionally, set key
then
OpenAIEmbeddings(document_model_name="MODEL_NAME", chunk_size=1)
A straight forward yet messy solution seems to be updating
validate_enviroment
validator function in langchain/embeddings/openai.py
by adding a check-and-fix. As this is where openai seem to be first provoked.
I'm suffering from these same issues, although my error message is a bit different (likely since I'm using LangChain v0.0.123:
2023-03-25 19:47:29.827 INFO openai: error_code=429 error_message='Requests to the Embeddings_Create Operation under Azure OpenAI API version 2022-12-01 have exceeded call rate limit of your current OpenAI S0 pricing tier. Please retry after 5 seconds. Please contact Azure support service if you would like to further increase the default rate limit.' error_param=None error_type=None message='OpenAI API error received' stream_error=False
2023-03-25 19:47:29.831 WARNING langchain.embeddings.openai: Retrying langchain.embeddings.openai.embed_with_retry.<locals>._completion_with_retry in 4.0 seconds as it raised RateLimitError: Requests to the Embeddings_Create Operation under Azure OpenAI API version 2022-12-01 have exceeded call rate limit of your current OpenAI S0 pricing tier. Please retry after 5 seconds. Please contact Azure support service if you would like to further increase the default rate limit..
It seems to me like the wait time between retries (which have clearly been handled in multiple places in LangChain) may not be the issues so much as the number of async workers sent to Azure in the space of a minute (since the limit is something like 300 per minute for embeddings). Does anyone know where in the LangChain code the number of max async workers is defined such that we could throttle it down to avoid this error perhaps?
I'm suffering from these same issues, although my error message is a bit different (likely since I'm using LangChain v0.0.123:
2023-03-25 19:47:29.827 INFO openai: error_code=429 error_message='Requests to the Embeddings_Create Operation under Azure OpenAI API version 2022-12-01 have exceeded call rate limit of your current OpenAI S0 pricing tier. Please retry after 5 seconds. Please contact Azure support service if you would like to further increase the default rate limit.' error_param=None error_type=None message='OpenAI API error received' stream_error=False 2023-03-25 19:47:29.831 WARNING langchain.embeddings.openai: Retrying langchain.embeddings.openai.embed_with_retry.<locals>._completion_with_retry in 4.0 seconds as it raised RateLimitError: Requests to the Embeddings_Create Operation under Azure OpenAI API version 2022-12-01 have exceeded call rate limit of your current OpenAI S0 pricing tier. Please retry after 5 seconds. Please contact Azure support service if you would like to further increase the default rate limit..
It seems to me like the wait time between retries (which have clearly been handled in multiple places in LangChain) may not be the issues so much as the number of async workers sent to Azure in the space of a minute (since the limit is something like 300 per minute for embeddings). Does anyone know where in the LangChain code the number of max async workers is defined such that we could throttle it down to avoid this error perhaps?
Don't really think this is the same issue, Maybe open a new issue page? I did encounter the same problem tho. My novel solution is to add a sleep inside the langchain library source code (I do not require high performance). I did not find a usage or performance limiter available to use.
Apologies, I copied the wrong error (this is a related one, but caused when you try to use embeddings in a vector store using llama-index
). My actual LangChain-only error is Too many inputs for model None. The max number of inputs is 1. We hope to increase the number of inputs per request soon. Please contact us through an Azure support request at: https://go.microsoft.com/fwlink/?linkid=2213926 for further questions.' error_param=None error_type=invalid_request_error message='OpenAI API error received' stream_error=False
@Granine where did you put the sleep call (at the top of the openai.py module perhaps; or maybe in the embedding loop)? I'm (currently) fine with something like that, but it seems like that just repeats the retry logic doesn't it? But if it works for you, I'm willing to give it a whirl!
I'm on langchain=0.0.119 but
OpenAIEmbeddings()
throws an AuthenticationError: Incorrect API key provided... it seems that it tries to authenticate through the OpenAI API instead of the AzureOpenAI service, even when I configured the OPENAI_API_TYPE and OPENAI_API_BASE previously. Does anyone have the same problem?... tried with version 0.0.117 but the problem persistsFor some reason, the suggested implementation on the documentation (by setting environmental variables) does not work. However, a workaround is before
openAIEmbedding
is ever called, importopenai
package and set parameters manually eg:import openai openai.api_base = "www.x.com/" openai.api_type = 'azure' openai.api_version = "2022-12-01" # optionally, set key
then
OpenAIEmbeddings(document_model_name="MODEL_NAME", chunk_size=1)
I'm on langchain=0.0.119 but
OpenAIEmbeddings()
throws an AuthenticationError: Incorrect API key provided... it seems that it tries to authenticate through the OpenAI API instead of the AzureOpenAI service, even when I configured the OPENAI_API_TYPE and OPENAI_API_BASE previously. Does anyone have the same problem?... tried with version 0.0.117 but the problem persistsFor some reason, the suggested implementation on the documentation (by setting environmental variables) does not work. However, a workaround is before
openAIEmbedding
is ever called, importopenai
package and set parameters manually eg:import openai openai.api_base = "www.x.com/" openai.api_type = 'azure' openai.api_version = "2022-12-01" # optionally, set key
then
OpenAIEmbeddings(document_model_name="MODEL_NAME", chunk_size=1)
it worked! thank you! @Granine
I've spent the last 5 hours or so trying to work this ❗💲 #️⃣ ⚡er out, which has certainly put a damper on my hackathon spirits :)
I'm using:
and I was just trying to following the simple indexing getting started guide
Setting chunk_size=1
against the OpenAIEmbeddings
initializer is indeed one part of the solve, at least until Azure opens up more chunks. It seems they have this as a filter in the router at your deployment level (or a part of the model ingestion handler) as you'll get this error The API deployment for this resource does not exist. If you created the deployment within the last 5 minutes, please wait a moment and try again.
with or without chunk_size=1
, but you'll only get Too many inputs for model None. The max number of inputs is 1. We hope to increase the number of inputs per request soon. Please contact us through an Azure support request at: https://go.microsoft.com/fwlink/?linkid=2213926 for further questions.
if you end up on a valid embedding model.
Embedding model is the key point here, if you try to use a model that outputs text rather than embedding vectors you'll get openai.error.InvalidRequestError: The embeddings operation does not work with the specified model, text-davinci-002. Please choose different model and try again. You can learn more about which models can be used with each operation here: https://go.microsoft.com/fwlink/?linkid=2197993.
So actually, if you are receiving Too many inputs for model None. The max number of inputs is 1. ...
then it's a sign your deployment is set up correctly in Azure and you're actually hitting it. Just put in chunk_size=1
and you'll hit our final boss: openai.error.InvalidRequestError: Must provide an 'engine' or 'deployment_id' parameter to create a <class 'openai.api_resources.completion.Completion'>
As a brief tangent, the easiest way to actually get this to work up until this point is to just make a deployment called text-embedding-ada-002
in Azure and deploy the model with the same name to it, then you're work with the langchain defaults as of the time of writing.
If you want to use another model then pass in OpenAIEmbeddings(model="<your embeddings deployment name>")
, (note not model_name
nor document_model_name
/query_model_name
unless you specifically want different models for upsert vs query, just model
. See here)
Hopefully MS adds a front door model
attribute => model router in the near future so things are API compatible with the openAI API.
Now, I digress. Onto M. Bison
In the particular case of the getting started guide I was using, the actual problem was that things were not being set up correctly to bring in a text LLM for the LLMChain that is behind the operations. I confirmed this by adding "engine":"text-davinci-003"
on the line below this (as well as having text-davinci-003 deployed to Azure with a deployment of the same name), which made everything work.
To 'properly' fix this you need to do as follows (assuming you have the right Azure env vars in .env):
from dotenv import load_dotenv
from langchain.document_loaders import DirectoryLoader
from langchain.embeddings.openai import OpenAIEmbeddings
from langchain.indexes import VectorstoreIndexCreator
from langchain.llms import OpenAI
load_dotenv()
# Instantiate a Langchain OpenAI class, but give it a default engine
llm = OpenAI(model_kwargs={'engine':'text-davinci-003'})
# Load docs
docs = DirectoryLoader(os.path.join(os.path.dirname(__file__), 'my_docs_directory'))
# Sort out the chunking issue
embedding_model = OpenAIEmbeddings(chunk_size=1)
# Load things into your index, this all works fine so long as you do the above. Things are auto instantiated to use Azure correctly, so long as you have a deployment of text-embedding-ada-002 with the same model in your Azure instance
index = VectorstoreIndexCreator(embedding=embedding_model).from_loaders([my_files])
# This is where the fix is. You can pass in the pre-instantiated LLM with the default text model set, so your LLMChain behind the query action actually works
index.query(ym_query, llm=llm)
But as you can see, this is case by case. I've poked a lot but I don't see a way to set the text LLM model against the high level abstractions (i.e. VectorstoreIndexCreator) in a way that propagates. Hopefully it's there and I'm just missing it.
So the easy solution is.... well, there's no easy solution. The problem is actually the default instantiation of the Azure based openAI LLM takes into account that it's running on Azure, but doesn't have nice defaults or ways to pass through that you need to change the engine as well/instead of the model. Really, this is just a huge pain for the library writers due to the decision to have 'deployments' on Azure and not adhere to the model
attribute like the OpenAI API
@EAYoshi I've been trying to play around with the code here: https://github.com/alphasecio/llama-index/blob/main/index.py to get it working on Azure, changed it to use from langchain.llms import AzureOpenAI
but not having any luck. Getting the same error as you, tried setting chunk size to 1 but all I get is this:
error_code=DeploymentNotFound error_message='The API deployment for this resource does not exist. If you created the deployment within the last 5 minutes, please wait a moment and try again.' error_param=None error_type=None message='OpenAI API error received' stream_error=False
Do you think this is related?
Likely, yes.
First, make sure these environment variables are set correctly in the context of your code
OPENAI_API_BASE=<azure OpenAI base URL without deployment>
OPENAI_API_KEY=<azure key>
OPENAI_API_TYPE=azure
Then, make sure you have an Azure deployment named text-davinci-003 that is set to use the text-davinci-003
model
Finally, try changing this line to be
llm_predictor = LLMPredictor(llm=OpenAI(model_kwargs={'engine':'text-davinci-003'}))
then play around with the temperature once it works, I'm not sure where that parameter should go
This issue is actually a bug in Azure, as the address to access is end_point/deployments/[Model deployment name]/embedding. The system failed to retrieve the name of the model deployment and instead retrieved the model name, which is text-embedding-ada-002. If your model deployment name is different from the model name, it will result in a "not found" error. Therefore, the solution is to rename your model deployment to text-embedding-ada-002. Happy Coding.
I've been stuck all day and this thread solved my problems. Thank everyone who contributed to this thread!
I'm also on Azure, Embeddings work in code like this:
embeddings = OpenAIEmbeddings(model="embed", chunk_size=1) vectorstore = FAISS.from_documents(pages, embeddings)
But it doesn't work when I'm trying to use the SemanticSimilarityExampleSelector:
example_selector = SemanticSimilarityExampleSelector.from_examples( examples, OpenAIEmbeddings(model="embed", chunk_size=1), FAISS, k=2 )
I'm getting the error:
KeyError: 'Could not automatically map embed to a tokeniser. Please use
tiktok.get_encodingto explicitly get the tokeniser you expect.'
Am I doing something wrong, is this a bug in langchain or on Azure?
(langchain version 0.0.135)
Hi, in Azure, is there any workaround to have a bigger chunk_size instead of chunk_size=1?
The problem with chunk_size=1 in Azure is performance. In native OpenAI API, we can set chunk_size=1000 for doing embedding., which is much more faster than the one with chunk_size=1 in Azure
I'm not sure if this helps, but I was porting AutoGPT to work on Azure. With a recent update it broke with a similar embedding issue. The devs were using openai.Embedding.create(input=[text], model="text-embedding-ada-002")["data"][0]["embedding"]
to create an embedding. The working Azure flavour for me was to use: openai.Embedding.create(input=[text], engine=cfg.openai_embedding_deployment_id)["data"][0]["embedding"]
where the deployment id for the embedding model is specified in .env
. Note the change from model
to engine
and engine
specifying the deployment id.
Hi, in Azure, is there any workaround to have a bigger chunk_size instead of chunk_size=1?
The problem with chunk_size=1 in Azure is performance. In native OpenAI API, we can set chunk_size=1000 for doing embedding., which is much more faster than the one with chunk_size=1 in Azure
As of time of writing, Azure only supports one chunk. The message you get back from the API says it's temporary and they're working on it, so I'm pretty sure it's server side and there's nothing to be done about it. Talk to your Azure TAM if you have one.
it seems the issue with model vs. deployment separation was reintroduced with the tiktoken tokenizer checks. I opened a PR to finally add proper support for Azure OpenAI embedding models by separating the deployment and model names: https://github.com/hwchase17/langchain/pull/3076
Resolved just using embeddings = OpenAIEmbeddings(model="text-embedding-ada-002", chunk_size=1)
for my case.
You can specify the Azure OpenAI Service deployment name since v0.0.146 OpenAIEmbeddings(deployment="your-azure-deployment-name", model="text-embedding-ada-002", chunk_size=1)
well it's clear that a base endpoint for your azure services is not defined.
what do you expect this line of code to do:
openai.api_base = os.getenv('')
@sai-kiran-adusumilli you also exposed your API key in your first post. You edited it, but any user can view the edit history. You need to trash that key now and get a new one... unless you want a huge Azure bill.
if i delete the comment ? can users still see it
well I get email notifications for comments and that is where I first saw you exposed your API key.... deleting the comment here isn't going to delete the email in my inbox.
@JonAtDocuWare I proposed #3711 to address this issue.
According to the docs: https://learn.microsoft.com/en-us/azure/cognitive-services/openai/reference#embeddings
The POST
request should look like:
POST https://{your-resource-name}.openai.azure.com/openai/deployments/{deployment-id}/embeddings?api-version={api-version}
However my current PR is not working because it generates
https://xxxxxx.openai.azure.com/openai/engines/text-embedding-ada-002/embeddings
I am currently debugging this problem.
Explicitly passing texts to embed as "input" resovled this for me. On openai==0.27.6
openai.Embedding.create(input=texts, model="text-embedding-ada-002")
I was having the same issue but using OpenAIEmbeddings(model="text-embedding-ada-002", chunk_size=1)
fixed it.
Now I'm getting the INFO:openai:error_code=429 error_message='Requests to the Embeddings_Create Operation under Azure OpenAI API version 2022-12-01 have exceeded call rate limit of your current OpenAI S0 pricing tier. Please retry after 1 second. Please go here: https://aka.ms/oai/quotaincrease if you would like to further increase the default rate limit.' error_param=None error_type=None message='OpenAI API error received' stream_error=False
error .
openai.Embedding.create(input=texts, model="text-embedding-ada-002")
This worked for me but had to add deployment_id
On openai==0.27.1
embedding =openai.Embedding.create(input="input text", model="text-embedding-ada-002",deployment_id="text-embedding-ada-002")
I was having the same issue but using
OpenAIEmbeddings(model="text-embedding-ada-002", chunk_size=1)
fixed it. Now I'm getting theINFO:openai:error_code=429 error_message='Requests to the Embeddings_Create Operation under Azure OpenAI API version 2022-12-01 have exceeded call rate limit of your current OpenAI S0 pricing tier. Please retry after 1 second. Please go here: https://aka.ms/oai/quotaincrease if you would like to further increase the default rate limit.' error_param=None error_type=None message='OpenAI API error received' stream_error=False
error .
The 429 error is not a bug in your code, is the API throttling you. Which Azure region are you using ? You could try using Azure OpenAI in a different region.
Thanks! I'm in west europe. I'll try that...
Edit: it worked on South Central US with no issues. Seems that in West Europe the API is throttling me indeed.
The prefect way to solve this problem is:
from langchain.document_loaders import TextLoader
from langchain.indexes import VectorstoreIndexCreator
from langchain.embeddings import OpenAIEmbeddings
loader = TextLoader('text.txt')
embeddings = OpenAIEmbeddings(model='text-embedding-ada-002',
deployment='your deployment name',
openai_api_base='https://your-resource-name.openai.azure.com',
openai_api_type='azure',
openai_api_key='your key',
chunk_size=1)
index = VectorstoreIndexCreator(embedding=embeddings).from_loaders([loader])
has been resolved, closing this issue.
I am getting error :-
Retrying langchain.embeddings.openai.embed_with_retry.<locals>._completion_with_retry in 4.0 seconds as it raised RateLimitError: You exceeded your current quota, please check your plan and billing details..
I am using this code :-
embeddings = OpenAIEmbeddings(
openai_api_key=OPENAI_API_KEY)
version
langchain==0.0.113
you only need this:
embeddings: OpenAIEmbeddings = OpenAIEmbeddings(
openai_api_base= f"https://{AZURE_OPENAI_SERVICE}.openai.azure.com",
openai_api_type='azure',
deployment='text-embedding-ada-002',
openai_api_key=AZURE_OPENAI_API_KEY,
chunk_size=1,
)
query_result = embeddings.embed_query("is this issue solve?")
My deployment name and model name are the same: text-embedding-ada-002
When using the AzureOpenAI LLM the OpenAIEmbeddings are not working. After reviewing source, I believe this is because the class does not accept any parameters other than an api_key. A "Model deployment name" parameter would be needed, since the model name alone is not enough to identify the engine. I did, however, find a workaround. If you name your deployment exactly "text-embedding-ada-002" then OpenAIEmbeddings will work.
edit: my workaround is working on version .088, but not the current version.