search

pinecone-io / pinecone-python-client

The Pinecone Python client
https://www.pinecone.io/docs
Apache License 2.0
284 stars 78 forks source link

[Bug] Collection model dimension limit should be 20000 as per the ui and not 2000 #366

Closed drivard closed 1 month ago

drivard commented 1 month ago

Hi,

I wrote code to automate the backup of our index using the Pinecone client and encountered an issue where the code throws the following exception:

pinecone.core.client.exceptions.PineconeApiValueError: Invalid value for `dimension`, must be a value less than or equal to `2000`

This exception occurs in both pc.list_collections() and pc.create_collection(name=collection_name, source=index_name). Despite the collection creation throwing the error, I can see the collection being created in the UI.

When creating a new collection, I noticed that the dimension limit is 20,000 and not 2,000 as specified in the code at this line https://github.com/pinecone-io/pinecone-python-client/blob/8b7e7de50e72248de683a24ada024cf18e6cdce3/pinecone/core/client/model/collection_model.py#L64-L69

image

When I edit my virtual environment package to use 20,000 as the dimension limit, the error no longer occurs.

Best regards

evantao96 commented 1 month ago

@drivard ,

The Pinecone Team has reproduced the issue you raised in our collections API and implemented a fix. We are targeting a release for the Pinecone Python client for next week.

Please let me know if you experience any other issues with our clients.

Best,

Evan

drivard commented 1 month ago

I will watch for the release and test it.

Thanks

jhamon commented 1 month ago

I just shipped a v5 release which should resolve this issue and add a few new features as well. The root cause was an error in the openapi specification we use to generate parts of the SDK, so the fix got baked in along with some other changes and didn't require a standalone PR to fix. Thanks for reporting.

https://github.com/pinecone-io/pinecone-python-client/releases/tag/v5.0.0