Closed dragos-cojocari closed 2 years ago
Here is a simple end to end way to reproduce this in a fresh Conda Env
% conda create -n test-pb python=3.9
conda activate test-pb
pip install google-ai-platform
python
from google.cloud import aiplatform
With output
% conda create -n test-pb python=3.9
Collecting package metadata (current_repodata.json): done
Solving environment: done
==> WARNING: A newer version of conda exists. <==
current version: 4.10.3
latest version: 4.12.0
Please update conda by running
$ conda update -n base -c defaults conda
## Package Plan ##
environment location: /Users/dragoscojocari/opt/anaconda3/envs/test-pb
added / updated specs:
- python=3.9
The following packages will be downloaded:
package | build
---------------------------|-----------------
ca-certificates-2022.4.26 | hecd8cb5_0 124 KB
certifi-2022.5.18.1 | py39hecd8cb5_0 148 KB
openssl-1.1.1o | hca72f7f_0 2.2 MB
sqlite-3.38.3 | h707629a_0 1.2 MB
tk-8.6.11 | h3fd3227_1 3.0 MB
xz-5.2.5 | hca72f7f_1 244 KB
zlib-1.2.12 | h4dc903c_2 94 KB
------------------------------------------------------------
Total: 7.0 MB
The following NEW packages will be INSTALLED:
ca-certificates pkgs/main/osx-64::ca-certificates-2022.4.26-hecd8cb5_0
certifi pkgs/main/osx-64::certifi-2022.5.18.1-py39hecd8cb5_0
libcxx pkgs/main/osx-64::libcxx-12.0.0-h2f01273_0
libffi pkgs/main/osx-64::libffi-3.3-hb1e8313_2
ncurses pkgs/main/osx-64::ncurses-6.3-hca72f7f_2
openssl pkgs/main/osx-64::openssl-1.1.1o-hca72f7f_0
pip pkgs/main/osx-64::pip-21.2.4-py39hecd8cb5_0
python pkgs/main/osx-64::python-3.9.12-hdfd78df_0
readline pkgs/main/osx-64::readline-8.1.2-hca72f7f_1
setuptools pkgs/main/osx-64::setuptools-61.2.0-py39hecd8cb5_0
sqlite pkgs/main/osx-64::sqlite-3.38.3-h707629a_0
tk pkgs/main/osx-64::tk-8.6.11-h3fd3227_1
tzdata pkgs/main/noarch::tzdata-2022a-hda174b7_0
wheel pkgs/main/noarch::wheel-0.37.1-pyhd3eb1b0_0
xz pkgs/main/osx-64::xz-5.2.5-hca72f7f_1
zlib pkgs/main/osx-64::zlib-1.2.12-h4dc903c_2
Proceed ([y]/n)? y
Downloading and Extracting Packages
tk-8.6.11 | 3.0 MB | ###################################################################################################### | 100%
xz-5.2.5 | 244 KB | ###################################################################################################### | 100%
openssl-1.1.1o | 2.2 MB | ###################################################################################################### | 100%
ca-certificates-2022 | 124 KB | ###################################################################################################### | 100%
certifi-2022.5.18.1 | 148 KB | ###################################################################################################### | 100%
sqlite-3.38.3 | 1.2 MB | ###################################################################################################### | 100%
zlib-1.2.12 | 94 KB | ###################################################################################################### | 100%
Preparing transaction: done
Verifying transaction: done
Executing transaction: done
#
# To activate this environment, use
#
# $ conda activate test-pb
#
# To deactivate an active environment, use
#
# $ conda deactivate
% conda activate test-pb
(test-pb) % pip install google-cloud-aiplatform
Collecting google-cloud-aiplatform
Downloading google_cloud_aiplatform-1.13.0-py2.py3-none-any.whl (1.8 MB)
|████████████████████████████████| 1.8 MB 1.5 MB/s
Collecting google-cloud-bigquery<3.0.0dev,>=1.15.0
Using cached google_cloud_bigquery-2.34.3-py2.py3-none-any.whl (206 kB)
Collecting google-cloud-resource-manager<3.0.0dev,>=1.3.3
Downloading google_cloud_resource_manager-1.5.0-py2.py3-none-any.whl (230 kB)
|████████████████████████████████| 230 kB 851 kB/s
Collecting google-cloud-storage<3.0.0dev,>=1.32.0
Using cached google_cloud_storage-2.3.0-py2.py3-none-any.whl (107 kB)
Collecting google-api-core[grpc]!=2.0.*,!=2.1.*,!=2.2.*,!=2.3.0,<3.0.0dev,>=1.31.5
Downloading google_api_core-2.8.0-py3-none-any.whl (114 kB)
|████████████████████████████████| 114 kB 586 kB/s
Collecting proto-plus>=1.15.0
Downloading proto_plus-1.20.4-py3-none-any.whl (46 kB)
|████████████████████████████████| 46 kB 1.9 MB/s
Collecting packaging>=14.3
Using cached packaging-21.3-py3-none-any.whl (40 kB)
Collecting protobuf>=3.12.0
Downloading protobuf-4.21.0-cp37-abi3-macosx_10_9_universal2.whl (483 kB)
|████████████████████████████████| 483 kB 1.8 MB/s
Collecting googleapis-common-protos<2.0dev,>=1.52.0
Downloading googleapis_common_protos-1.56.1-py2.py3-none-any.whl (211 kB)
|████████████████████████████████| 211 kB 1.2 MB/s
Collecting requests<3.0.0dev,>=2.18.0
Using cached requests-2.27.1-py2.py3-none-any.whl (63 kB)
Collecting google-auth<3.0dev,>=1.25.0
Using cached google_auth-2.6.6-py2.py3-none-any.whl (156 kB)
Collecting grpcio-status<2.0dev,>=1.33.2
Downloading grpcio_status-1.46.3-py3-none-any.whl (10.0 kB)
Collecting grpcio<2.0dev,>=1.33.2
Downloading grpcio-1.46.3-cp39-cp39-macosx_10_10_x86_64.whl (4.4 MB)
|████████████████████████████████| 4.4 MB 1.4 MB/s
Collecting cachetools<6.0,>=2.0.0
Using cached cachetools-5.1.0-py3-none-any.whl (9.2 kB)
Collecting rsa<5,>=3.1.4
Using cached rsa-4.8-py3-none-any.whl (39 kB)
Collecting pyasn1-modules>=0.2.1
Using cached pyasn1_modules-0.2.8-py2.py3-none-any.whl (155 kB)
Collecting six>=1.9.0
Using cached six-1.16.0-py2.py3-none-any.whl (11 kB)
Collecting google-cloud-core<3.0.0dev,>=1.4.1
Using cached google_cloud_core-2.3.0-py2.py3-none-any.whl (29 kB)
Collecting google-resumable-media<3.0dev,>=0.6.0
Downloading google_resumable_media-2.3.3-py2.py3-none-any.whl (76 kB)
|████████████████████████████████| 76 kB 2.1 MB/s
Collecting python-dateutil<3.0dev,>=2.7.2
Using cached python_dateutil-2.8.2-py2.py3-none-any.whl (247 kB)
Collecting grpc-google-iam-v1<1.0.0dev,>=0.12.3
Using cached grpc_google_iam_v1-0.12.4-py2.py3-none-any.whl (26 kB)
Collecting google-crc32c<2.0dev,>=1.0
Using cached google_crc32c-1.3.0-cp39-cp39-macosx_10_9_x86_64.whl (30 kB)
Collecting pyparsing!=3.0.5,>=2.0.2
Downloading pyparsing-3.0.9-py3-none-any.whl (98 kB)
|████████████████████████████████| 98 kB 1.3 MB/s
Collecting pyasn1<0.5.0,>=0.4.6
Using cached pyasn1-0.4.8-py2.py3-none-any.whl (77 kB)
Collecting charset-normalizer~=2.0.0
Using cached charset_normalizer-2.0.12-py3-none-any.whl (39 kB)
Requirement already satisfied: certifi>=2017.4.17 in /Users/dragoscojocari/opt/anaconda3/envs/test-pb/lib/python3.9/site-packages (from requests<3.0.0dev,>=2.18.0->google-api-core[grpc]!=2.0.*,!=2.1.*,!=2.2.*,!=2.3.0,<3.0.0dev,>=1.31.5->google-cloud-aiplatform) (2022.5.18.1)
Collecting idna<4,>=2.5
Using cached idna-3.3-py3-none-any.whl (61 kB)
Collecting urllib3<1.27,>=1.21.1
Using cached urllib3-1.26.9-py2.py3-none-any.whl (138 kB)
Installing collected packages: pyasn1, urllib3, six, rsa, pyasn1-modules, protobuf, idna, charset-normalizer, cachetools, requests, grpcio, googleapis-common-protos, google-auth, pyparsing, grpcio-status, google-crc32c, google-api-core, python-dateutil, proto-plus, packaging, grpc-google-iam-v1, google-resumable-media, google-cloud-core, google-cloud-storage, google-cloud-resource-manager, google-cloud-bigquery, google-cloud-aiplatform
Successfully installed cachetools-5.1.0 charset-normalizer-2.0.12 google-api-core-2.8.0 google-auth-2.6.6 google-cloud-aiplatform-1.13.0 google-cloud-bigquery-2.34.3 google-cloud-core-2.3.0 google-cloud-resource-manager-1.5.0 google-cloud-storage-2.3.0 google-crc32c-1.3.0 google-resumable-media-2.3.3 googleapis-common-protos-1.56.1 grpc-google-iam-v1-0.12.4 grpcio-1.46.3 grpcio-status-1.46.3 idna-3.3 packaging-21.3 proto-plus-1.20.4 protobuf-4.21.0 pyasn1-0.4.8 pyasn1-modules-0.2.8 pyparsing-3.0.9 python-dateutil-2.8.2 requests-2.27.1 rsa-4.8 six-1.16.0 urllib3-1.26.9
(test-pb) % python
Python 3.9.12 (main, Apr 5 2022, 01:53:17)
[Clang 12.0.0 ] :: Anaconda, Inc. on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> from google.cloud import aiplatform
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/Users/dragoscojocari/opt/anaconda3/envs/test-pb/lib/python3.9/site-packages/google/cloud/aiplatform/__init__.py", line 41, in <module>
from google.cloud.aiplatform.matching_engine import (
File "/Users/dragoscojocari/opt/anaconda3/envs/test-pb/lib/python3.9/site-packages/google/cloud/aiplatform/matching_engine/__init__.py", line 26, in <module>
from google.cloud.aiplatform.matching_engine.matching_engine_index_endpoint import (
File "/Users/dragoscojocari/opt/anaconda3/envs/test-pb/lib/python3.9/site-packages/google/cloud/aiplatform/matching_engine/matching_engine_index_endpoint.py", line 30, in <module>
from google.cloud.aiplatform.matching_engine._protos import match_service_pb2
File "/Users/dragoscojocari/opt/anaconda3/envs/test-pb/lib/python3.9/site-packages/google/cloud/aiplatform/matching_engine/_protos/match_service_pb2.py", line 54, in <module>
_descriptor.FieldDescriptor(
File "/Users/dragoscojocari/opt/anaconda3/envs/test-pb/lib/python3.9/site-packages/google/protobuf/descriptor.py", line 560, in __new__
_message.Message._CheckCalledFromGeneratedFile()
TypeError: Descriptors cannot not be created directly.
If this call came from a _pb2.py file, your generated code is out of date and must be regenerated with protoc >= 3.19.0.
If you cannot immediately regenerate your protos, some other possible workarounds are:
1. Downgrade the protobuf package to 3.20.x or lower.
2. Set PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION=python (but this will use pure-Python parsing and will be much slower).
More information: https://developers.google.com/protocol-buffers/docs/news/2022-05-06#python-updates
>>>
This started occurring today, May 26th after protobuf 4.21 has been launched. Trying to explicitly set protobuf version to 3.21.0 in the requirements file of the project importing aiplatform does not help either. The other Google Python APIs ( storage etc) do not seem affected by this.
Environment details
google-cloud-aiplatform
version: 1.13.0Steps to reproduce
Build any Python code that imports google.aiplatform
Code example
Stack trace