googleapis / google-cloud-python

Google Cloud Client Library for Python
https://googleapis.github.io/google-cloud-python/
Apache License 2.0
4.86k stars 1.53k forks source link

container: container_v1 not importable - dependency issue? #5043

Closed hawksight closed 6 years ago

hawksight commented 6 years ago

Hello, I've been trying to install and play with google-cloud-container library but I am having issues following the quickstart as described here.

Every time I try to: from google.cloud import container_v1 I get: DistributionNotFound: The 'google-api-core' distribution was not found and is required by the application

I use gcloud CLI every day and the right API is enabled in the project I am working with.

I've tried in several different virtual envs, hence reproduce steps are pretty simple. Seem some similar issues on googling but not related this this library. Any help appreciated.

  1. OS type and version : MacOS High Sierra - 10.13.3
  2. Python version and virtual environment information python --version : Python 3.6.4
  3. google-cloud-python version pip show google-cloud, pip show google-<service> or pip freeze
    (venv) ➜  scripts git:(gke-1-9) ✗ pip show google-cloud-container
    Name: google-cloud-container
    Version: 0.1.1
    Summary: Google Container Engine API client library
    Home-page: https://github.com/GoogleCloudPlatform/google-cloud-python
    Author: Google LLC
    Author-email: googleapis-packages@google.com
    License: Apache 2.0
    Location: /Users/pfiddes/Projects/kubernetes/venv/lib/python3.6/site-packages
    Requires: google-api-core

freeze on my actual env, not a brand new one (which doesn't work either)

(venv) ➜  scripts git:(gke-1-9) ✗ pip freeze
appnope==0.1.0
asn1crypto==0.24.0
boto3==1.5.22
botocore==1.8.36
cachetools==2.0.1
certifi==2018.1.18
cffi==1.11.4
chardet==3.0.4
click==6.7
colorama==0.3.9
colorclass==2.2.0
credstash==1.14.0
cryptography==2.0
decorator==4.2.1
docopt==0.6.2
docutils==0.14
google-api-core==1.0.0
google-auth==1.3.0
google-cloud-container==0.1.1
googleapis-common-protos==1.5.3
grpcio==1.10.0
httplib2==0.10.3
idna==2.6
ipaddress==1.0.19
ipython==6.2.1
ipython-genutils==0.2.0
jedi==0.11.1
Jinja2==2.10
jmespath==0.9.3
kubernetes==4.0.0
MarkupSafe==1.0
oauth2client==4.1.2
oauthlib==2.0.6
packaging==16.8
parso==0.1.1
pexpect==4.3.1
pickleshare==0.7.4
pip-upgrader==1.4.4
plumbum==1.6.5
prompt-toolkit==1.0.15
protobuf==3.5.2
ptyprocess==0.5.2
pyasn1==0.4.2
pyasn1-modules==0.2.1
pycparser==2.18
Pygments==2.2.0
pyparsing==2.2.0
python-dateutil==2.6.1
pytz==2018.3
PyYAML==3.12
requests==2.18.4
requests-oauthlib==0.8.0
rsa==3.4.2
s3transfer==0.1.12
simplegeneric==0.8.1
six==1.11.0
terminaltables==3.1.0
traitlets==4.3.2
urllib3==1.22
wcwidth==0.1.7
websocket-client==0.46.0
  1. Stacktrace if available - Ran from ipython:
    
    In [1]: from google.cloud import container_v1
    ---------------------------------------------------------------------------
    DistributionNotFound                      Traceback (most recent call last)
    <ipython-input-1-6685c094e0c1> in <module>()
    ----> 1 from google.cloud import container_v1

~/Projects/kubernetes/venv/lib/python3.6/site-packages/google/cloud/container_v1/init.py in () 15 from future import absolute_import 16 ---> 17 from google.cloud.container_v1 import types 18 from google.cloud.container_v1.gapic import cluster_manager_client 19 from google.cloud.container_v1.gapic import enums

~/Projects/kubernetes/venv/lib/python3.6/site-packages/google/cloud/container_v1/types.py in () 16 import sys 17 ---> 18 from google.api_core.protobuf_helpers import get_messages 19 20 from google.api import http_pb2

~/Projects/kubernetes/venv/lib/python3.6/site-packages/google/api_core/init.py in () 21 22 ---> 23 version = get_distribution('google-api-core').version

/usr/local/lib/python3.6/site-packages/pkg_resources/init.py in get_distribution(dist) 577 dist = Requirement.parse(dist) 578 if isinstance(dist, Requirement): --> 579 dist = get_provider(dist) 580 if not isinstance(dist, Distribution): 581 raise TypeError("Expected string, Requirement, or Distribution", dist)

/usr/local/lib/python3.6/site-packages/pkg_resources/init.py in get_provider(moduleOrReq) 449 """Return an IResourceProvider for the named module or requirement""" 450 if isinstance(moduleOrReq, Requirement): --> 451 return working_set.find(moduleOrReq) or require(str(moduleOrReq))[0] 452 try: 453 module = sys.modules[moduleOrReq]

/usr/local/lib/python3.6/site-packages/pkg_resources/init.py in require(self, *requirements) 997 included, even if they were already activated in this working set. 998 """ --> 999 needed = self.resolve(parse_requirements(requirements)) 1000 1001 for dist in needed:

/usr/local/lib/python3.6/site-packages/pkg_resources/init.py in resolve(self, requirements, env, installer, replace_conflicting, extras) 883 if dist is None: 884 requirers = required_by.get(req, None) --> 885 raise DistributionNotFound(req, requirers) 886 to_activate.append(dist) 887 if dist not in req:

DistributionNotFound: The 'google-api-core' distribution was not found and is required by the application


6. Steps to reproduce

python3 -m venv venv source venv/bin/activate pip3 install -U pip setuptools wheel pip3 install google-cloud-container pip3 install ipython ipython

run: from google.cloud import container_v1

pip3 install google-cloud ipython3

run: from google.cloud import container_v1

Always get error:

In [1]: from google.cloud import Container_v1

ImportError Traceback (most recent call last)

in () ----> 1 from google.cloud import Container_v1 ImportError: cannot import name 'Container_v1' In [2]: from google.cloud import container_v1 --------------------------------------------------------------------------- DistributionNotFound Traceback (most recent call last) in () ----> 1 from google.cloud import container_v1 ~/temp/venv/lib/python3.6/site-packages/google/cloud/container_v1/__init__.py in () 15 from __future__ import absolute_import 16 ---> 17 from google.cloud.container_v1 import types 18 from google.cloud.container_v1.gapic import cluster_manager_client 19 from google.cloud.container_v1.gapic import enums ~/temp/venv/lib/python3.6/site-packages/google/cloud/container_v1/types.py in () 16 import sys 17 ---> 18 from google.api_core.protobuf_helpers import get_messages 19 20 from google.api import http_pb2 ~/temp/venv/lib/python3.6/site-packages/google/api_core/__init__.py in () 21 22 ---> 23 __version__ = get_distribution('google-api-core').version /usr/local/lib/python3.6/site-packages/pkg_resources/__init__.py in get_distribution(dist) 577 dist = Requirement.parse(dist) 578 if isinstance(dist, Requirement): --> 579 dist = get_provider(dist) 580 if not isinstance(dist, Distribution): 581 raise TypeError("Expected string, Requirement, or Distribution", dist) /usr/local/lib/python3.6/site-packages/pkg_resources/__init__.py in get_provider(moduleOrReq) 449 """Return an IResourceProvider for the named module or requirement""" 450 if isinstance(moduleOrReq, Requirement): --> 451 return working_set.find(moduleOrReq) or require(str(moduleOrReq))[0] 452 try: 453 module = sys.modules[moduleOrReq] /usr/local/lib/python3.6/site-packages/pkg_resources/__init__.py in require(self, *requirements) 997 included, even if they were already activated in this working set. 998 """ --> 999 needed = self.resolve(parse_requirements(requirements)) 1000 1001 for dist in needed: /usr/local/lib/python3.6/site-packages/pkg_resources/__init__.py in resolve(self, requirements, env, installer, replace_conflicting, extras) 883 if dist is None: 884 requirers = required_by.get(req, None) --> 885 raise DistributionNotFound(req, requirers) 886 to_activate.append(dist) 887 if dist not in req: DistributionNotFound: The 'google-api-core' distribution was not found and is required by the application ``` Freeze on a clean env: ``` appnope==0.1.0 cachetools==2.0.1 certifi==2018.1.18 chardet==3.0.4 decorator==4.2.1 dill==0.2.7.1 future==0.16.0 gapic-google-cloud-datastore-v1==0.15.3 gapic-google-cloud-error-reporting-v1beta1==0.15.3 gapic-google-cloud-logging-v2==0.91.3 google-api-core==0.1.4 google-auth==1.4.1 google-cloud==0.32.0 google-cloud-bigquery==0.28.0 google-cloud-bigquery-datatransfer==0.1.1 google-cloud-bigtable==0.28.1 google-cloud-container==0.1.1 google-cloud-core==0.28.1 google-cloud-datastore==1.4.0 google-cloud-dns==0.28.0 google-cloud-error-reporting==0.28.0 google-cloud-firestore==0.28.0 google-cloud-language==1.0.1 google-cloud-logging==1.4.0 google-cloud-monitoring==0.28.1 google-cloud-pubsub==0.30.1 google-cloud-resource-manager==0.28.1 google-cloud-runtimeconfig==0.28.1 google-cloud-spanner==0.29.0 google-cloud-speech==0.30.0 google-cloud-storage==1.6.0 google-cloud-trace==0.17.0 google-cloud-translate==1.3.1 google-cloud-videointelligence==1.0.1 google-cloud-vision==0.29.0 google-gax==0.15.16 google-resumable-media==0.3.1 googleapis-common-protos==1.5.3 grpc-google-iam-v1==0.11.4 grpcio==1.10.0 httplib2==0.10.3 idna==2.6 ipython==6.2.1 ipython-genutils==0.2.0 jedi==0.11.1 oauth2client==3.0.0 parso==0.1.1 pexpect==4.4.0 pickleshare==0.7.4 ply==3.8 prompt-toolkit==1.0.15 proto-google-cloud-datastore-v1==0.90.4 proto-google-cloud-error-reporting-v1beta1==0.15.3 proto-google-cloud-logging-v2==0.91.3 protobuf==3.5.2 psutil==5.4.3 ptyprocess==0.5.2 pyasn1==0.4.2 pyasn1-modules==0.2.1 Pygments==2.2.0 pytz==2018.3 requests==2.18.4 rsa==3.4.2 simplegeneric==0.8.1 six==1.11.0 traitlets==4.3.2 urllib3==1.22 wcwidth==0.1.7 ``` Some pip shows: ``` (venv) ➜ temp pip3 show google-cloud-container Name: google-cloud-container Version: 0.1.1 Summary: Google Container Engine API client library Home-page: https://github.com/GoogleCloudPlatform/google-cloud-python Author: Google LLC Author-email: googleapis-packages@google.com License: Apache 2.0 Location: /Users/pfiddes/temp/venv/lib/python3.6/site-packages Requires: google-api-core (venv) ➜ temp pip3 show google-api-core Name: google-api-core Version: 0.1.4 Summary: Core Google API Client Library Home-page: https://github.com/GoogleCloudPlatform/google-cloud-python Author: Google Cloud Platform Author-email: googleapis-publisher@google.com License: Apache 2.0 Location: /Users/pfiddes/temp/venv/lib/python3.6/site-packages Requires: requests, googleapis-common-protos, protobuf, pytz, setuptools, google-auth, six (venv) ➜ temp pip3 show google-cloud Name: google-cloud Version: 0.32.0 Summary: API Client library for Google Cloud Home-page: https://github.com/GoogleCloudPlatform/google-cloud-python Author: Google Cloud Platform Author-email: googleapis-publisher@google.com License: Apache 2.0 Location: /Users/pfiddes/temp/venv/lib/python3.6/site-packages Requires: google-cloud-core, google-cloud-bigquery, google-cloud-trace, google-cloud-runtimeconfig, google-api-core, google-cloud-speech, google-cloud-spanner, google-cloud-pubsub, google-cloud-translate, google-cloud-vision, google-cloud-resource-manager, google-cloud-bigtable, google-cloud-videointelligence, google-cloud-monitoring, google-cloud-container, google-cloud-datastore, google-cloud-dns, google-cloud-storage, google-cloud-language, google-cloud-firestore, google-cloud-error-reporting, google-cloud-logging, google-cloud-bigquery-datatransfer ``` 7. Code example - None yet, can't import the package
tseaver commented 6 years ago

Hmm, trying to reproduce (I don't use activate because it can lead to PEBCAK errors):

$ /opt/Python-3.6.3/bin/python3 -m venv /tmp/gcp-5043
$ /tmp/gcp-5043/bin/pip install -U pip setuptools wheel
Requirement already up-to-date: pip in /tmp/gcp-5043/lib/python3.6/site-packages
Collecting setuptools
  Using cached setuptools-38.5.2-py2.py3-none-any.whl
Collecting wheel
  Using cached wheel-0.30.0-py2.py3-none-any.whl
Installing collected packages: setuptools, wheel
  Found existing installation: setuptools 28.8.0
    Uninstalling setuptools-28.8.0:
      Successfully uninstalled setuptools-28.8.0
Successfully installed setuptools-38.5.2 wheel-0.30.0
$ /tmp/gcp-5043/bin/pip install google-cloud-container
Collecting google-cloud-container
  Downloading google_cloud_container-0.1.1-py2.py3-none-any.whl (50kB)
    100% |████████████████████████████████| 51kB 896kB/s 
Collecting google-api-core[grpc]<2.0.0dev,>=0.1.0 (from google-cloud-container)
  Using cached google_api_core-1.1.0-py2.py3-none-any.whl
...
Successfully installed cachetools-2.0.1 certifi-2018.1.18 chardet-3.0.4 google-api-core-1.1.0 google-auth-1.4.1 google-cloud-container-0.1.1 googleapis-common-protos-1.5.3 grpcio-1.10.0 idna-2.6 protobuf-3.5.2 pyasn1-0.4.2 pyasn1-modules-0.2.1 pytz-2018.3 requests-2.18.4 rsa-3.4.2 six-1.11.0 urllib3-1.22
$ /tmp/gcp-5043/bin/pip list
DEPRECATION: The default format will switch to columns in the future. You can use --format=(legacy|columns) (or define a format=(legacy|columns) in your pip.conf under the [list] section) to disable this warning.
cachetools (2.0.1)
certifi (2018.1.18)
chardet (3.0.4)
google-api-core (1.1.0)
google-auth (1.4.1)
google-cloud-container (0.1.1)
googleapis-common-protos (1.5.3)
grpcio (1.10.0)
idna (2.6)
pip (9.0.1)
protobuf (3.5.2)
pyasn1 (0.4.2)
pyasn1-modules (0.2.1)
pytz (2018.3)
requests (2.18.4)
rsa (3.4.2)
setuptools (38.5.2)
six (1.11.0)
urllib3 (1.22)
wheel (0.30.0)
$ /tmp/gcp-5043/bin/python
Python 3.6.3 (default, Oct  4 2017, 16:14:25) 
[GCC 4.8.4] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from google.cloud import container_v1
>>> 

Likewise under ipython:

$ /tmp/gcp-5043/bin/pip install ipython
Collecting ipython
...
Successfully installed decorator-4.2.1 ipython-6.2.1 ipython-genutils-0.2.0 jedi-0.11.1 parso-0.1.1 pexpect-4.4.0 pickleshare-0.7.4 prompt-toolkit-1.0.15 ptyprocess-0.5.2 pygments-2.2.0 simplegeneric-0.8.1 traitlets-4.3.2 wcwidth-0.1.7
$ /tmp/gcp-5043/bin/ipython
/tmp/gcp-5043/lib/python3.6/site-packages/IPython/paths.py:53: UserWarning: Moving ~/.config/ipython to ~/.ipython
  warn('Moving {0} to {1}'.format(cu(xdg_ipdir), cu(ipdir)))
Python 3.6.3 (default, Oct  4 2017, 16:14:25) 
Type 'copyright', 'credits' or 'license' for more information
IPython 6.2.1 -- An enhanced Interactive Python. Type '?' for help.

In [1]: from google.cloud import container_v1

In [2]:
hawksight commented 6 years ago

I've just tried again following the above - worked. Also then went back to my venv - worked. Apologies for raising the issue, it must have been a PEBCAK issue.

But thanks for the tip, I'll not be using activate so much.

tseaver commented 6 years ago

I'm glad you are able to use the library!

hawksight commented 6 years ago

Actually thinking about it and incase anyone else hits the same bump. I think ipython had loaded from a global install rather than the virtual environment that was active:

/usr/local/lib/python3.6/site-packages/IPython/core/interactiveshell.py:728: UserWarning: Attempting to work in a virtualenv. If you encounter problems, please install IPython inside the virtualenv.
  warn("Attempting to work in a virtualenv. If you encounter problems, please "
Python 3.6.4 (default, Mar  9 2018, 23:15:03)
Type 'copyright', 'credits' or 'license' for more information
IPython 6.2.1 -- An enhanced Interactive Python. Type '?' for help.

Funnily enough the warning was there but I just didn't see it.