search

googleapis / python-bigquery

Apache License 2.0
730 stars 297 forks source link

Needs protobuf minimum version 3.12.0 #305

Closed anistark closed 3 years ago

anistark commented 3 years ago

If you are still having issues, please be sure to include as much information as possible:

Environment details

Steps to reproduce

  1. Just install the package, import somewhere and run it.

Code example

from google.cloud import bigquery

Stack trace

File "/.../venv/lib/python3.8/site-packages/google/protobuf/internal/python_message.py", line 570, in _GetFieldByName
    return message_descriptor.fields_by_name[field_name]
KeyError: 'proto3_optional'

...

   from google.cloud import bigquery
  File "/.../venv/lib/python3.8/site-packages/google/cloud/bigquery/__init__.py", line 35, in <module>
    from google.cloud.bigquery.client import Client
  File "/.../venv/lib/python3.8/site-packages/google/cloud/bigquery/client.py", line 57, in <module>
    from google.cloud.bigquery import _pandas_helpers
  File "/.../venv/lib/python3.8/site-packages/google/cloud/bigquery/_pandas_helpers.py", line 36, in <module>
    from google.cloud.bigquery import schema
  File "/.../venv/lib/python3.8/site-packages/google/cloud/bigquery/schema.py", line 19, in <module>
    from google.cloud.bigquery_v2 import types
  File "/.../venv/lib/python3.8/site-packages/google/cloud/bigquery_v2/__init__.py", line 19, in <module>
    from .types.encryption_config import EncryptionConfiguration
  File "/.../venv/lib/python3.8/site-packages/google/cloud/bigquery_v2/types/__init__.py", line 18, in <module>
    from .encryption_config import EncryptionConfiguration
  File "/.../venv/lib/python3.8/site-packages/google/cloud/bigquery_v2/types/encryption_config.py", line 29, in <module>
    class EncryptionConfiguration(proto.Message):
  File "/.../venv/lib/python3.8/site-packages/proto/message.py", line 215, in __new__
    field=[i.descriptor for i in fields],
  File "/.../venv/lib/python3.8/site-packages/proto/message.py", line 215, in <listcomp>
    field=[i.descriptor for i in fields],
  File "/.../venv/lib/python3.8/site-packages/proto/fields.py", line 104, in descriptor
    self._descriptor = descriptor_pb2.FieldDescriptorProto(
  File "/.../venv/lib/python3.8/site-packages/google/protobuf/internal/python_message.py", line 509, in init
    field = _GetFieldByName(message_descriptor, field_name)
  File "/.../venv/lib/python3.8/site-packages/google/protobuf/internal/python_message.py", line 572, in _GetFieldByName
    raise ValueError('Protocol message %s has no "%s" field.' %
ValueError: Protocol message FieldDescriptorProto has no "proto3_optional" field.
HemangChothani commented 3 years ago

@anistark google-cloud-bigquery has dependency of proto-plus as you can see below which use the protobuf. https://github.com/googleapis/python-bigquery/blob/d1eb8b3dcc789916c5d3ba8464f62b1f8bef35ff/setup.py#L33

proto-plus library has already mentioned a minimum version of protobuf, you can find here https://github.com/googleapis/proto-plus-python/blob/d045cbf058cbb8f4ca98dd06741270fcaee865be/setup.py#L39

anistark commented 3 years ago

And yet it somehow didn't install for me.

HemangChothani commented 3 years ago

@anistark i have tried to install with fresh env and it's working fine, even i tried to install proto-plus and 'google-cloud-bigquery' with already installed older version of protobuf, it successfully uninstall the older version and install the latest version of library.

aylusltd commented 3 years ago

Clean install on AWS Sagemaker CondaPython3 Kernel. Looks like same bug.

Collecting google-cloud-bigquery
  Downloading google_cloud_bigquery-2.6.1-py2.py3-none-any.whl (211 kB)
     |████████████████████████████████| 211 kB 15.9 MB/s eta 0:00:01
Requirement already satisfied: six<2.0.0dev,>=1.13.0 in /home/ec2-user/anaconda3/envs/python3/lib/python3.6/site-packages (from google-cloud-bigquery) (1.14.0)
Collecting proto-plus>=1.10.0
  Downloading proto-plus-1.13.0.tar.gz (44 kB)
     |████████████████████████████████| 44 kB 3.5 MB/s  eta 0:00:01
Collecting google-resumable-media<2.0dev,>=0.6.0
  Downloading google_resumable_media-1.2.0-py2.py3-none-any.whl (75 kB)
     |████████████████████████████████| 75 kB 6.2 MB/s  eta 0:00:01
Requirement already satisfied: protobuf>=3.12.0 in /home/ec2-user/anaconda3/envs/python3/lib/python3.6/site-packages (from google-cloud-bigquery) (3.12.2)
Collecting google-cloud-core<2.0dev,>=1.4.1
  Downloading google_cloud_core-1.5.0-py2.py3-none-any.whl (27 kB)
Collecting google-api-core[grpc]<2.0.0dev,>=1.23.0
  Downloading google_api_core-1.24.1-py2.py3-none-any.whl (92 kB)
     |████████████████████████████████| 92 kB 14.7 MB/s eta 0:00:01
Collecting google-crc32c<2.0dev,>=1.0; python_version >= "3.5"
  Downloading google_crc32c-1.1.0-cp36-cp36m-manylinux2010_x86_64.whl (39 kB)
Requirement already satisfied: setuptools in /home/ec2-user/anaconda3/envs/python3/lib/python3.6/site-packages (from protobuf>=3.12.0->google-cloud-bigquery) (46.1.3.post20200330)
Collecting googleapis-common-protos<2.0dev,>=1.6.0
  Downloading googleapis_common_protos-1.52.0-py2.py3-none-any.whl (100 kB)
     |████████████████████████████████| 100 kB 13.2 MB/s ta 0:00:01
Requirement already satisfied: requests<3.0.0dev,>=2.18.0 in /home/ec2-user/anaconda3/envs/python3/lib/python3.6/site-packages (from google-api-core[grpc]<2.0.0dev,>=1.23.0->google-cloud-bigquery) (2.23.0)
Requirement already satisfied: pytz in /home/ec2-user/anaconda3/envs/python3/lib/python3.6/site-packages (from google-api-core[grpc]<2.0.0dev,>=1.23.0->google-cloud-bigquery) (2019.3)
Collecting google-auth<2.0dev,>=1.21.1
  Downloading google_auth-1.24.0-py2.py3-none-any.whl (114 kB)
     |████████████████████████████████| 114 kB 68.9 MB/s eta 0:00:01
Collecting grpcio<2.0dev,>=1.29.0; extra == "grpc"
  Downloading grpcio-1.34.0-cp36-cp36m-manylinux2014_x86_64.whl (4.0 MB)
     |████████████████████████████████| 4.0 MB 62.5 MB/s eta 0:00:01
Requirement already satisfied: cffi>=1.0.0 in /home/ec2-user/anaconda3/envs/python3/lib/python3.6/site-packages (from google-crc32c<2.0dev,>=1.0; python_version >= "3.5"->google-resumable-media<2.0dev,>=0.6.0->google-cloud-bigquery) (1.14.0)
Requirement already satisfied: idna<3,>=2.5 in /home/ec2-user/anaconda3/envs/python3/lib/python3.6/site-packages (from requests<3.0.0dev,>=2.18.0->google-api-core[grpc]<2.0.0dev,>=1.23.0->google-cloud-bigquery) (2.9)
Requirement already satisfied: urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1 in /home/ec2-user/anaconda3/envs/python3/lib/python3.6/site-packages (from requests<3.0.0dev,>=2.18.0->google-api-core[grpc]<2.0.0dev,>=1.23.0->google-cloud-bigquery) (1.25.8)
Requirement already satisfied: certifi>=2017.4.17 in /home/ec2-user/anaconda3/envs/python3/lib/python3.6/site-packages (from requests<3.0.0dev,>=2.18.0->google-api-core[grpc]<2.0.0dev,>=1.23.0->google-cloud-bigquery) (2020.4.5.1)
Requirement already satisfied: chardet<4,>=3.0.2 in /home/ec2-user/anaconda3/envs/python3/lib/python3.6/site-packages (from requests<3.0.0dev,>=2.18.0->google-api-core[grpc]<2.0.0dev,>=1.23.0->google-cloud-bigquery) (3.0.4)
Collecting pyasn1-modules>=0.2.1
  Downloading pyasn1_modules-0.2.8-py2.py3-none-any.whl (155 kB)
     |████████████████████████████████| 155 kB 64.1 MB/s eta 0:00:01
Collecting cachetools<5.0,>=2.0.0
  Downloading cachetools-4.2.0-py3-none-any.whl (12 kB)
Requirement already satisfied: rsa<5,>=3.1.4; python_version >= "3.6" in /home/ec2-user/anaconda3/envs/python3/lib/python3.6/site-packages (from google-auth<2.0dev,>=1.21.1->google-api-core[grpc]<2.0.0dev,>=1.23.0->google-cloud-bigquery) (3.4.2)
Requirement already satisfied: pycparser in /home/ec2-user/anaconda3/envs/python3/lib/python3.6/site-packages (from cffi>=1.0.0->google-crc32c<2.0dev,>=1.0; python_version >= "3.5"->google-resumable-media<2.0dev,>=0.6.0->google-cloud-bigquery) (2.20)
Requirement already satisfied: pyasn1<0.5.0,>=0.4.6 in /home/ec2-user/anaconda3/envs/python3/lib/python3.6/site-packages (from pyasn1-modules>=0.2.1->google-auth<2.0dev,>=1.21.1->google-api-core[grpc]<2.0.0dev,>=1.23.0->google-cloud-bigquery) (0.4.8)
Building wheels for collected packages: proto-plus
  Building wheel for proto-plus (setup.py) ... done
  Created wheel for proto-plus: filename=proto_plus-1.13.0-py3-none-any.whl size=41591 sha256=185a880a276da72cb83ea7d6a1abf2b33be68ff357c6fe11e2b324d313f683ba
  Stored in directory: /home/ec2-user/.cache/pip/wheels/5a/76/e1/dd2dc8192e9a6b687d8c7550acc7c8edf059a4f16296e7ea89
Successfully built proto-plus
Installing collected packages: proto-plus, google-crc32c, google-resumable-media, googleapis-common-protos, pyasn1-modules, cachetools, google-auth, grpcio, google-api-core, google-cloud-core, google-cloud-bigquery
Successfully installed cachetools-4.2.0 google-api-core-1.24.1 google-auth-1.24.0 google-cloud-bigquery-2.6.1 google-cloud-core-1.5.0 google-crc32c-1.1.0 google-resumable-media-1.2.0 googleapis-common-protos-1.52.0 grpcio-1.34.0 proto-plus-1.13.0 pyasn1-modules-0.2.8
WARNING: You are using pip version 20.0.2; however, version 20.3.3 is available.
You should consider upgrading via the '/home/ec2-user/anaconda3/envs/python3/bin/python -m pip install --upgrade pip' command.
ValueError                                Traceback (most recent call last)
<ipython-input-4-b3914349486e> in <module>
----> 1 from google.cloud import bigquery
      2 query_string = """
      3 SELECT * FROM `<---REDACTED DB NAME--->` LIMIT 10
      4 """
      5 client = bigquery.Client()

~/anaconda3/envs/python3/lib/python3.6/site-packages/google/cloud/bigquery/__init__.py in <module>
     33 __version__ = bigquery_version.__version__
     34 
---> 35 from google.cloud.bigquery.client import Client
     36 from google.cloud.bigquery.dataset import AccessEntry
     37 from google.cloud.bigquery.dataset import Dataset

~/anaconda3/envs/python3/lib/python3.6/site-packages/google/cloud/bigquery/client.py in <module>
     55 from google.cloud.bigquery._helpers import _verify_job_config_type
     56 from google.cloud.bigquery._http import Connection
---> 57 from google.cloud.bigquery import _pandas_helpers
     58 from google.cloud.bigquery.dataset import Dataset
     59 from google.cloud.bigquery.dataset import DatasetListItem

~/anaconda3/envs/python3/lib/python3.6/site-packages/google/cloud/bigquery/_pandas_helpers.py in <module>
     34     pyarrow = None
     35 
---> 36 from google.cloud.bigquery import schema
     37 
     38 

~/anaconda3/envs/python3/lib/python3.6/site-packages/google/cloud/bigquery/schema.py in <module>
     17 from six.moves import collections_abc
     18 
---> 19 from google.cloud.bigquery_v2 import types
     20 
     21 

~/anaconda3/envs/python3/lib/python3.6/site-packages/google/cloud/bigquery_v2/__init__.py in <module>
     17 
     18 
---> 19 from .types.encryption_config import EncryptionConfiguration
     20 from .types.model import DeleteModelRequest
     21 from .types.model import GetModelRequest

~/anaconda3/envs/python3/lib/python3.6/site-packages/google/cloud/bigquery_v2/types/__init__.py in <module>
     16 #
     17 
---> 18 from .encryption_config import EncryptionConfiguration
     19 from .model_reference import ModelReference
     20 from .standard_sql import (

~/anaconda3/envs/python3/lib/python3.6/site-packages/google/cloud/bigquery_v2/types/encryption_config.py in <module>
     27 
     28 
---> 29 class EncryptionConfiguration(proto.Message):
     30     r"""
     31 

~/anaconda3/envs/python3/lib/python3.6/site-packages/proto/message.py in __new__(mcls, name, bases, attrs)
    212         desc = descriptor_pb2.DescriptorProto(
    213             name=name,
--> 214             field=[i.descriptor for i in fields],
    215             oneof_decl=[
    216                 descriptor_pb2.OneofDescriptorProto(name=i) for i in oneofs.keys()

~/anaconda3/envs/python3/lib/python3.6/site-packages/proto/message.py in <listcomp>(.0)
    212         desc = descriptor_pb2.DescriptorProto(
    213             name=name,
--> 214             field=[i.descriptor for i in fields],
    215             oneof_decl=[
    216                 descriptor_pb2.OneofDescriptorProto(name=i) for i in oneofs.keys()

~/anaconda3/envs/python3/lib/python3.6/site-packages/proto/fields.py in descriptor(self)
    109                 type_name=type_name,
    110                 json_name=self.json_name,
--> 111                 proto3_optional=self.optional,
    112             )
    113 

ValueError: Protocol message FieldDescriptorProto has no "proto3_optional" field.
anzen-l commented 3 years ago

Clean install on AWS Sagemaker CondaPython3 Kernel. Looks like same bug.

Collecting google-cloud-bigquery
  Downloading google_cloud_bigquery-2.6.1-py2.py3-none-any.whl (211 kB)
     |████████████████████████████████| 211 kB 15.9 MB/s eta 0:00:01
Requirement already satisfied: six<2.0.0dev,>=1.13.0 in /home/ec2-user/anaconda3/envs/python3/lib/python3.6/site-packages (from google-cloud-bigquery) (1.14.0)
Collecting proto-plus>=1.10.0
  Downloading proto-plus-1.13.0.tar.gz (44 kB)
     |████████████████████████████████| 44 kB 3.5 MB/s  eta 0:00:01
Collecting google-resumable-media<2.0dev,>=0.6.0
  Downloading google_resumable_media-1.2.0-py2.py3-none-any.whl (75 kB)
     |████████████████████████████████| 75 kB 6.2 MB/s  eta 0:00:01
Requirement already satisfied: protobuf>=3.12.0 in /home/ec2-user/anaconda3/envs/python3/lib/python3.6/site-packages (from google-cloud-bigquery) (3.12.2)
Collecting google-cloud-core<2.0dev,>=1.4.1
  Downloading google_cloud_core-1.5.0-py2.py3-none-any.whl (27 kB)
Collecting google-api-core[grpc]<2.0.0dev,>=1.23.0
  Downloading google_api_core-1.24.1-py2.py3-none-any.whl (92 kB)
     |████████████████████████████████| 92 kB 14.7 MB/s eta 0:00:01
Collecting google-crc32c<2.0dev,>=1.0; python_version >= "3.5"
  Downloading google_crc32c-1.1.0-cp36-cp36m-manylinux2010_x86_64.whl (39 kB)
Requirement already satisfied: setuptools in /home/ec2-user/anaconda3/envs/python3/lib/python3.6/site-packages (from protobuf>=3.12.0->google-cloud-bigquery) (46.1.3.post20200330)
Collecting googleapis-common-protos<2.0dev,>=1.6.0
  Downloading googleapis_common_protos-1.52.0-py2.py3-none-any.whl (100 kB)
     |████████████████████████████████| 100 kB 13.2 MB/s ta 0:00:01
Requirement already satisfied: requests<3.0.0dev,>=2.18.0 in /home/ec2-user/anaconda3/envs/python3/lib/python3.6/site-packages (from google-api-core[grpc]<2.0.0dev,>=1.23.0->google-cloud-bigquery) (2.23.0)
Requirement already satisfied: pytz in /home/ec2-user/anaconda3/envs/python3/lib/python3.6/site-packages (from google-api-core[grpc]<2.0.0dev,>=1.23.0->google-cloud-bigquery) (2019.3)
Collecting google-auth<2.0dev,>=1.21.1
  Downloading google_auth-1.24.0-py2.py3-none-any.whl (114 kB)
     |████████████████████████████████| 114 kB 68.9 MB/s eta 0:00:01
Collecting grpcio<2.0dev,>=1.29.0; extra == "grpc"
  Downloading grpcio-1.34.0-cp36-cp36m-manylinux2014_x86_64.whl (4.0 MB)
     |████████████████████████████████| 4.0 MB 62.5 MB/s eta 0:00:01
Requirement already satisfied: cffi>=1.0.0 in /home/ec2-user/anaconda3/envs/python3/lib/python3.6/site-packages (from google-crc32c<2.0dev,>=1.0; python_version >= "3.5"->google-resumable-media<2.0dev,>=0.6.0->google-cloud-bigquery) (1.14.0)
Requirement already satisfied: idna<3,>=2.5 in /home/ec2-user/anaconda3/envs/python3/lib/python3.6/site-packages (from requests<3.0.0dev,>=2.18.0->google-api-core[grpc]<2.0.0dev,>=1.23.0->google-cloud-bigquery) (2.9)
Requirement already satisfied: urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1 in /home/ec2-user/anaconda3/envs/python3/lib/python3.6/site-packages (from requests<3.0.0dev,>=2.18.0->google-api-core[grpc]<2.0.0dev,>=1.23.0->google-cloud-bigquery) (1.25.8)
Requirement already satisfied: certifi>=2017.4.17 in /home/ec2-user/anaconda3/envs/python3/lib/python3.6/site-packages (from requests<3.0.0dev,>=2.18.0->google-api-core[grpc]<2.0.0dev,>=1.23.0->google-cloud-bigquery) (2020.4.5.1)
Requirement already satisfied: chardet<4,>=3.0.2 in /home/ec2-user/anaconda3/envs/python3/lib/python3.6/site-packages (from requests<3.0.0dev,>=2.18.0->google-api-core[grpc]<2.0.0dev,>=1.23.0->google-cloud-bigquery) (3.0.4)
Collecting pyasn1-modules>=0.2.1
  Downloading pyasn1_modules-0.2.8-py2.py3-none-any.whl (155 kB)
     |████████████████████████████████| 155 kB 64.1 MB/s eta 0:00:01
Collecting cachetools<5.0,>=2.0.0
  Downloading cachetools-4.2.0-py3-none-any.whl (12 kB)
Requirement already satisfied: rsa<5,>=3.1.4; python_version >= "3.6" in /home/ec2-user/anaconda3/envs/python3/lib/python3.6/site-packages (from google-auth<2.0dev,>=1.21.1->google-api-core[grpc]<2.0.0dev,>=1.23.0->google-cloud-bigquery) (3.4.2)
Requirement already satisfied: pycparser in /home/ec2-user/anaconda3/envs/python3/lib/python3.6/site-packages (from cffi>=1.0.0->google-crc32c<2.0dev,>=1.0; python_version >= "3.5"->google-resumable-media<2.0dev,>=0.6.0->google-cloud-bigquery) (2.20)
Requirement already satisfied: pyasn1<0.5.0,>=0.4.6 in /home/ec2-user/anaconda3/envs/python3/lib/python3.6/site-packages (from pyasn1-modules>=0.2.1->google-auth<2.0dev,>=1.21.1->google-api-core[grpc]<2.0.0dev,>=1.23.0->google-cloud-bigquery) (0.4.8)
Building wheels for collected packages: proto-plus
  Building wheel for proto-plus (setup.py) ... done
  Created wheel for proto-plus: filename=proto_plus-1.13.0-py3-none-any.whl size=41591 sha256=185a880a276da72cb83ea7d6a1abf2b33be68ff357c6fe11e2b324d313f683ba
  Stored in directory: /home/ec2-user/.cache/pip/wheels/5a/76/e1/dd2dc8192e9a6b687d8c7550acc7c8edf059a4f16296e7ea89
Successfully built proto-plus
Installing collected packages: proto-plus, google-crc32c, google-resumable-media, googleapis-common-protos, pyasn1-modules, cachetools, google-auth, grpcio, google-api-core, google-cloud-core, google-cloud-bigquery
Successfully installed cachetools-4.2.0 google-api-core-1.24.1 google-auth-1.24.0 google-cloud-bigquery-2.6.1 google-cloud-core-1.5.0 google-crc32c-1.1.0 google-resumable-media-1.2.0 googleapis-common-protos-1.52.0 grpcio-1.34.0 proto-plus-1.13.0 pyasn1-modules-0.2.8
WARNING: You are using pip version 20.0.2; however, version 20.3.3 is available.
You should consider upgrading via the '/home/ec2-user/anaconda3/envs/python3/bin/python -m pip install --upgrade pip' command.
ValueError                                Traceback (most recent call last)
<ipython-input-4-b3914349486e> in <module>
----> 1 from google.cloud import bigquery
      2 query_string = """
      3 SELECT * FROM `<---REDACTED DB NAME--->` LIMIT 10
      4 """
      5 client = bigquery.Client()

~/anaconda3/envs/python3/lib/python3.6/site-packages/google/cloud/bigquery/__init__.py in <module>
     33 __version__ = bigquery_version.__version__
     34 
---> 35 from google.cloud.bigquery.client import Client
     36 from google.cloud.bigquery.dataset import AccessEntry
     37 from google.cloud.bigquery.dataset import Dataset

~/anaconda3/envs/python3/lib/python3.6/site-packages/google/cloud/bigquery/client.py in <module>
     55 from google.cloud.bigquery._helpers import _verify_job_config_type
     56 from google.cloud.bigquery._http import Connection
---> 57 from google.cloud.bigquery import _pandas_helpers
     58 from google.cloud.bigquery.dataset import Dataset
     59 from google.cloud.bigquery.dataset import DatasetListItem

~/anaconda3/envs/python3/lib/python3.6/site-packages/google/cloud/bigquery/_pandas_helpers.py in <module>
     34     pyarrow = None
     35 
---> 36 from google.cloud.bigquery import schema
     37 
     38 

~/anaconda3/envs/python3/lib/python3.6/site-packages/google/cloud/bigquery/schema.py in <module>
     17 from six.moves import collections_abc
     18 
---> 19 from google.cloud.bigquery_v2 import types
     20 
     21 

~/anaconda3/envs/python3/lib/python3.6/site-packages/google/cloud/bigquery_v2/__init__.py in <module>
     17 
     18 
---> 19 from .types.encryption_config import EncryptionConfiguration
     20 from .types.model import DeleteModelRequest
     21 from .types.model import GetModelRequest

~/anaconda3/envs/python3/lib/python3.6/site-packages/google/cloud/bigquery_v2/types/__init__.py in <module>
     16 #
     17 
---> 18 from .encryption_config import EncryptionConfiguration
     19 from .model_reference import ModelReference
     20 from .standard_sql import (

~/anaconda3/envs/python3/lib/python3.6/site-packages/google/cloud/bigquery_v2/types/encryption_config.py in <module>
     27 
     28 
---> 29 class EncryptionConfiguration(proto.Message):
     30     r"""
     31 

~/anaconda3/envs/python3/lib/python3.6/site-packages/proto/message.py in __new__(mcls, name, bases, attrs)
    212         desc = descriptor_pb2.DescriptorProto(
    213             name=name,
--> 214             field=[i.descriptor for i in fields],
    215             oneof_decl=[
    216                 descriptor_pb2.OneofDescriptorProto(name=i) for i in oneofs.keys()

~/anaconda3/envs/python3/lib/python3.6/site-packages/proto/message.py in <listcomp>(.0)
    212         desc = descriptor_pb2.DescriptorProto(
    213             name=name,
--> 214             field=[i.descriptor for i in fields],
    215             oneof_decl=[
    216                 descriptor_pb2.OneofDescriptorProto(name=i) for i in oneofs.keys()

~/anaconda3/envs/python3/lib/python3.6/site-packages/proto/fields.py in descriptor(self)
    109                 type_name=type_name,
    110                 json_name=self.json_name,
--> 111                 proto3_optional=self.optional,
    112             )
    113 

ValueError: Protocol message FieldDescriptorProto has no "proto3_optional" field.

I have the same problem with big query & jupyter notebooks

Tsakunelson commented 3 years ago

Hi All, I am still having this same error, though have all the dependencies installed on a Jupyter Notebook in AWS Sagemaker including:


Protobuf = 3.13.0```

For Google BigQuery 2.12.0

But I get the following error:

```---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-3-017a44a94d83> in <module>
----> 1 df = gbq.read_gbq('SELECT * FROM `cmg-consumer-analytics.122142903.ga_sessions_20210314` LIMIT 10', project_id='cmg-consumer-analytics')

~/anaconda3/envs/python3/lib/python3.6/site-packages/pandas/io/gbq.py in read_gbq(query, project_id, index_col, col_order, reauth, auth_local_webserver, dialect, location, configuration, credentials, use_bqstorage_api, private_key, verbose, progress_bar_type)
    161     DataFrame.to_gbq : Write a DataFrame to Google BigQuery.
    162     """
--> 163     pandas_gbq = _try_import()
    164 
    165     kwargs: Dict[str, Union[str, bool]] = {}

~/anaconda3/envs/python3/lib/python3.6/site-packages/pandas/io/gbq.py in _try_import()
     15         "See the docs: https://pandas-gbq.readthedocs.io."
     16     )
---> 17     pandas_gbq = import_optional_dependency("pandas_gbq", extra=msg)
     18     return pandas_gbq
     19 

~/anaconda3/envs/python3/lib/python3.6/site-packages/pandas/compat/_optional.py in import_optional_dependency(name, extra, raise_on_missing, on_version)
     87     )
     88     try:
---> 89         module = importlib.import_module(name)
     90     except ImportError:
     91         if raise_on_missing:

~/anaconda3/envs/python3/lib/python3.6/importlib/__init__.py in import_module(name, package)
    124                 break
    125             level += 1
--> 126     return _bootstrap._gcd_import(name[level:], package, level)
    127 
    128 

~/anaconda3/envs/python3/lib/python3.6/importlib/_bootstrap.py in _gcd_import(name, package, level)

~/anaconda3/envs/python3/lib/python3.6/importlib/_bootstrap.py in _find_and_load(name, import_)

~/anaconda3/envs/python3/lib/python3.6/importlib/_bootstrap.py in _find_and_load_unlocked(name, import_)

~/anaconda3/envs/python3/lib/python3.6/importlib/_bootstrap.py in _load_unlocked(spec)

~/anaconda3/envs/python3/lib/python3.6/importlib/_bootstrap_external.py in exec_module(self, module)

~/anaconda3/envs/python3/lib/python3.6/importlib/_bootstrap.py in _call_with_frames_removed(f, *args, **kwds)

~/anaconda3/envs/python3/lib/python3.6/site-packages/pandas_gbq/__init__.py in <module>
----> 1 from .gbq import to_gbq, read_gbq, Context, context  # noqa
      2 
      3 from ._version import get_versions
      4 
      5 versions = get_versions()

~/anaconda3/envs/python3/lib/python3.6/site-packages/pandas_gbq/gbq.py in <module>
     10 try:
     11     from google.api_core import exceptions as google_exceptions
---> 12     from google.cloud import bigquery
     13 except ImportError:  # pragma: NO COVER
     14     bigquery = None

~/anaconda3/envs/python3/lib/python3.6/site-packages/google/cloud/bigquery/__init__.py in <module>
     33 __version__ = bigquery_version.__version__
     34 
---> 35 from google.cloud.bigquery.client import Client
     36 from google.cloud.bigquery.dataset import AccessEntry
     37 from google.cloud.bigquery.dataset import Dataset

~/anaconda3/envs/python3/lib/python3.6/site-packages/google/cloud/bigquery/client.py in <module>
     54 from google.cloud.bigquery._helpers import _verify_job_config_type
     55 from google.cloud.bigquery._http import Connection
---> 56 from google.cloud.bigquery import _pandas_helpers
     57 from google.cloud.bigquery.dataset import Dataset
     58 from google.cloud.bigquery.dataset import DatasetListItem

~/anaconda3/envs/python3/lib/python3.6/site-packages/google/cloud/bigquery/_pandas_helpers.py in <module>
     34     pyarrow = None
     35 
---> 36 from google.cloud.bigquery import schema
     37 
     38 

~/anaconda3/envs/python3/lib/python3.6/site-packages/google/cloud/bigquery/schema.py in <module>
     17 import collections
     18 
---> 19 from google.cloud.bigquery_v2 import types
     20 
     21 

~/anaconda3/envs/python3/lib/python3.6/site-packages/google/cloud/bigquery_v2/__init__.py in <module>
     17 
     18 
---> 19 from .types.encryption_config import EncryptionConfiguration
     20 from .types.model import DeleteModelRequest
     21 from .types.model import GetModelRequest

~/anaconda3/envs/python3/lib/python3.6/site-packages/google/cloud/bigquery_v2/types/__init__.py in <module>
     16 #
     17 
---> 18 from .encryption_config import EncryptionConfiguration
     19 from .model import (
     20     DeleteModelRequest,

~/anaconda3/envs/python3/lib/python3.6/site-packages/google/cloud/bigquery_v2/types/encryption_config.py in <module>
     27 
     28 
---> 29 class EncryptionConfiguration(proto.Message):
     30     r"""
     31 

~/anaconda3/envs/python3/lib/python3.6/site-packages/proto/message.py in __new__(mcls, name, bases, attrs)
    212         desc = descriptor_pb2.DescriptorProto(
    213             name=name,
--> 214             field=[i.descriptor for i in fields],
    215             oneof_decl=[
    216                 descriptor_pb2.OneofDescriptorProto(name=i) for i in oneofs.keys()

~/anaconda3/envs/python3/lib/python3.6/site-packages/proto/message.py in <listcomp>(.0)
    212         desc = descriptor_pb2.DescriptorProto(
    213             name=name,
--> 214             field=[i.descriptor for i in fields],
    215             oneof_decl=[
    216                 descriptor_pb2.OneofDescriptorProto(name=i) for i in oneofs.keys()

~/anaconda3/envs/python3/lib/python3.6/site-packages/proto/fields.py in descriptor(self)
    109                 type_name=type_name,
    110                 json_name=self.json_name,
--> 111                 proto3_optional=self.optional,
    112             )
    113 

ValueError: Protocol message FieldDescriptorProto has no "proto3_optional" field.```

Why does the error still prevail? am I missing something else? Feedback appreciated.