search

googleapis / google-cloud-python

Google Cloud Client Library for Python
https://googleapis.github.io/google-cloud-python/
Apache License 2.0
4.83k stars 1.52k forks source link

Storage.Client() looking for _http? #3813

Closed bw4sz closed 7 years ago

bw4sz commented 7 years ago

I can see several issues related to http requests, but its difficult to tell if this is the same root causes. Opening new issue

I was working on an older version of google-cloud on a compute engine, and there was no problem. Installing locally os brand new osx/python 2.7 I get the odd behavior.

from google.cloud import storage

#set service account credentials
os.environ["GOOGLE_APPLICATION_CREDENTIALS"] = "/Users/Ben/Dropbox/Google/MeerkatReader-9fbf10d1e30c.json"

storage_client=storage.Client()
Traceback (most recent call last):
  Python Shell, prompt 10, line 1
  File "/Library/Python/2.7/site-packages/google/cloud/storage/client.py", line 59, in __init__
    _http=_http)
TypeError: __init__() got an unexpected keyword argument '_http'

Previously, I did not need to generate my own authenticated http request. 

Bens-MBP:DeepMeerkat ben$ pip show google-cloud
Name: google-cloud
Version: 0.27.0
Summary: API Client library for Google Cloud
Home-page: https://github.com/GoogleCloudPlatform/google-cloud-python
Author: Google Cloud Platform
Author-email: googleapis-publisher@google.com
License: Apache 2.0
Location: /Library/Python/2.7/site-packages
Requires: google-cloud-error-reporting, google-cloud-runtimeconfig, google-cloud-spanner, google-cloud-vision, google-cloud-bigtable, google-cloud-monitoring, google-cloud-pubsub, google-cloud-videointelligence, google-cloud-dns, google-cloud-speech, google-cloud-bigquery, google-cloud-resource-manager, google-cloud-translate, google-cloud-datastore, google-cloud-storage, google-cloud-core, google-cloud-logging, google-cloud-language
dhermes commented 7 years ago

@bw4sz Can you do a pip show google-cloud-storage google-cloud-core requests httplib2?

bw4sz commented 7 years ago 
Bens-MBP:DeepMeerkat ben$ pip show google-cloud-storage google-cloud-core requests httplib2
Name: google-cloud-storage
Version: 1.3.1
Summary: Python Client for Google Cloud Storage
Home-page: https://github.com/GoogleCloudPlatform/google-cloud-python
Author: Google Cloud Platform
Author-email: googleapis-publisher@google.com
License: Apache 2.0
Location: /Library/Python/2.7/site-packages
Requires: requests, google-cloud-core, google-auth, google-resumable-media
---
Name: google-cloud-core
Version: 0.26.0
Summary: API Client library for Google Cloud: Core Helpers
Home-page: https://github.com/GoogleCloudPlatform/google-cloud-python
Author: Google Cloud Platform
Author-email: googleapis-publisher@google.com
License: Apache 2.0
Location: /Library/Python/2.7/site-packages
Requires: protobuf, tenacity, requests, google-auth, googleapis-common-protos, six
---
Name: requests
Version: 2.18.3
Summary: Python HTTP for Humans.
Home-page: http://python-requests.org
Author: Kenneth Reitz
Author-email: me@kennethreitz.org
License: Apache 2.0
Location: /Library/Python/2.7/site-packages
Requires: idna, certifi, urllib3, chardet
---
Name: httplib2
Version: 0.10.3
Summary: A comprehensive HTTP client library.
Home-page: https://github.com/httplib2/httplib2
Author: Joe Gregorio
Author-email: joe@bitworking.org
License: MIT
Location: /Library/Python/2.7/site-packages
Requires:
dhermes commented 7 years ago

@bw4sz ISTM you have all the correct versions.

My current best guess is that you're getting a version of google-cloud-core somewhere up higher on your import path. Try the following:

>>> from google.cloud import client
>>> print(client.__file__)
bw4sz commented 7 years ago

is this not the expected behavior?

from google.cloud import client
print(client.__file__)
/Library/Python/2.7/site-packages/google/cloud/client.pyc
dhermes commented 7 years ago

There isn't really any "expected behavior" here (packaging set-ups vary widely). However, this looks like it's coming from the right place, so my current best guess is debunked.

bw4sz commented 7 years ago

Alright, but your sense is that I somehow have an older version? Let me pip uninstall and use the curl method.

bw4sz commented 7 years ago

Reverting to 0.26.0 seems to work, I will reopen if needed. Thank you.

after: sudo pip install --upgrade google-cloud==0.26.0

from google.cloud import storage
import os
from google.cloud import storage

#set service account credentials
os.environ["GOOGLE_APPLICATION_CREDENTIALS"] = "/Users/Ben/Dropbox/Google/MeerkatReader-9fbf10d1e30c.json"

storage_client=storage.Client()

storage_client
<google.cloud.storage.client.Client object at 0x1112db610>
Bens-MBP:DeepMeerkat ben$ pip show google-cloud-storage google-cloud-core requests httplib2
Name: google-cloud-storage
Version: 1.3.1
Summary: Python Client for Google Cloud Storage
Home-page: https://github.com/GoogleCloudPlatform/google-cloud-python
Author: Google Cloud Platform
Author-email: googleapis-publisher@google.com
License: Apache 2.0
Location: /Library/Python/2.7/site-packages
Requires: requests, google-cloud-core, google-auth, google-resumable-media
---
Name: google-cloud-core
Version: 0.25.0
Summary: API Client library for Google Cloud: Core Helpers
Home-page: https://github.com/GoogleCloudPlatform/google-cloud-python
Author: Google Cloud Platform
Author-email: jjg+google-cloud-python@google.com
License: Apache 2.0
Location: /Library/Python/2.7/site-packages
Requires: protobuf, httplib2, google-auth, googleapis-common-protos, google-auth-httplib2, six
---
Name: requests
Version: 2.18.3
Summary: Python HTTP for Humans.
Home-page: http://python-requests.org
Author: Kenneth Reitz
Author-email: me@kennethreitz.org
License: Apache 2.0
Location: /Library/Python/2.7/site-packages
Requires: urllib3, idna, certifi, chardet
---
Name: httplib2
Version: 0.10.3
Summary: A comprehensive HTTP client library.
Home-page: https://github.com/httplib2/httplib2
Author: Joe Gregorio
Author-email: joe@bitworking.org
License: MIT
Location: /Library/Python/2.7/site-packages
Requires:
dhermes commented 7 years ago

@bw4sz As a coda to this, I would recommend against using google-cloud unless you really want 14+ packages. Instead depend on the explicit packages you want, e.g. google-cloud-storage.

RubensZimbres commented 3 years ago

Same problem here:

Traceback (most recent call last): File "<stdin>", line 1, in <module> File "/home/rubens_/.local/lib/python3.7/site-packages/google/cloud/storage/client.py", line 73, in __init__ environment. TypeError: __init__() got an unexpected keyword argument '_http'

Pip show:

`WARNING: pip is being invoked by an old script wrapper. This will fail in a future version of pip. Please see https://github.com/pypa/pip/issues/5599 for advice on fixing the underlying issue. To avoid this problem you can invoke Python with '-m pip' instead of running pip directly. Name: google-cloud-storage Version: 1.36.2 Summary: Google Cloud Storage API client library Home-page: https://github.com/googleapis/python-storage Author: Google LLC Author-email: googleapis-packages@google.com License: Apache 2.0 Location: /home/rubens_zimbres_intellimetricom/.local/lib/python3.7/site-packages Requires: google-auth, google-cloud-core, requests, google-resumable-media Required-by: fairing

Name: google-cloud-core Version: 1.6.0 Summary: Google Cloud API client core library Home-page: https://github.com/googleapis/python-cloud-core Author: Google LLC Author-email: googleapis-packages@google.com License: Apache 2.0 Location: /home/rubens_zimbres_intellimetricom/.local/lib/python3.7/site-packages Requires: google-api-core, google-auth, six Required-by: google-cloud-vision, google-cloud-translate, google-cloud-trace, google-cloud-storage, google-cloud-speech, google-cloud-spanner, google-cloud-runtimeconfig, google-cloud-resource-manager, google-cloud-pubsub, google-cloud-monitoring, google-cloud-logging, google-cloud-language, google-cloud-firestore, google-cloud-error-reporting, google-cloud-dns, google-cloud-datastore, google-cloud-bigtable, google-cloud-bigquery

Name: requests Version: 2.25.1 Summary: Python HTTP for Humans. Home-page: https://requests.readthedocs.io Author: Kenneth Reitz Author-email: me@kennethreitz.org License: Apache 2.0 Location: /home/rubens_zimbres_intellimetricom/.local/lib/python3.7/site-packages Requires: chardet, certifi, urllib3, idna Required-by: transformers, python-resize-image, moviepy, huggingface-hub, Google-Images-Search, google-gax, google-colab, google-cloud-storage, google-api-core, datasets, torchtext, tensorflow-datasets, tensorboard, Sphinx, spacy, smart-open, requests-oauthlib, pytorch-pretrained-bert, papermill, pandas-datareader, notebook-executor, nbdime, kubernetes, jupyter-aihub-deploy-extension, gcsfs, fastai, fairing, docker, datalab, cookiecutter, conda, anaconda-project, anaconda-client

Name: httplib2 Version: 0.19.0 Summary: A comprehensive HTTP client library. Home-page: https://github.com/httplib2/httplib2 Author: Joe Gregorio Author-email: joe@bitworking.org License: MIT Location: /home/rubens_zimbres_intellimetricom/.local/lib/python3.7/site-packages Requires: pyparsing Required-by: oauth2client, google-api-python-client, google-auth-httplib2, fairing, datalab`

Pip list:

absl-py 0.9.0 aiohttp 3.7.4 alabaster 0.7.12 anaconda-client 1.7.2 anaconda-project 0.8.3 ansiwrap 0.8.4 arrow 0.15.2 asn1crypto 0.24.0 astor 0.8.0 astroid 2.4.2 astropy 3.2.1 async-timeout 3.0.1 atomicwrites 1.3.0 attrs 19.1.0 audioread 2.1.8 Babel 2.7.0 backcall 0.1.0 backports.os 0.1.1 backports.shutil-get-terminal-size 1.0.0 bcolz 1.2.1 beautifulsoup4 4.8.0 bert 2.2.0 binaryornot 0.4.4 bitarray 1.0.1 bkcharts 0.2 bleach 3.1.0 blinker 1.4 blis 0.2.4 bokeh 1.3.4 boto 2.49.0 boto3 1.10.16 botocore 1.13.16 Bottleneck 1.2.1 bz2file 0.98 cachetools 3.1.1 certifi 2020.12.5 cffi 1.12.3 chardet 3.0.4 Click 7.0 cloudpickle 1.2.2 clyent 1.2.2 colorama 0.4.1 conda 4.9.2 conda-package-handling 1.3.11 configparser 4.0.2 confuse 1.0.0 contextlib2 0.6.0 cookiecutter 1.6.0 crepe 0.0.11 cryptography 2.7 cycler 0.10.0 cymem 2.0.2 Cython 0.29.13 cytoolz 0.10.0 dask 2.5.0 dataclasses 0.6 datalab 1.1.5 datasets 1.4.1 DateTime 4.3 de-core-news-sm 2.1.0 decorator 4.4.0 defusedxml 0.6.0 dialogflow 1.1.0 dill 0.2.9 distributed 2.5.1 dlib 19.20.0 docker 4.1.0 docutils 0.15.2 en-core-web-sm 2.1.0 entrypoints 0.3 enum34 1.1.6 erlastic 2.0.0 et-xmlfile 1.0.1 fairing 0.5.3 fastai 1.0.59 fastcache 1.1.0 fastprogress 0.1.21 filelock 3.0.12 Flask 1.1.1 fsspec 0.5.1 future 0.16.0 fuzzywuzzy 0.17.0 gapic-google-cloud-datastore-v1 0.15.3 gapic-google-cloud-error-reporting-v1beta1 0.15.3 gapic-google-cloud-logging-v2 0.91.3 gapic-google-cloud-pubsub-v1 0.15.4 gapic-google-cloud-spanner-admin-database-v1 0.15.3 gapic-google-cloud-spanner-admin-instance-v1 0.15.3 gapic-google-cloud-spanner-v1 0.15.3 gapic-google-cloud-speech-v1 0.15.3 gapic-google-cloud-speech-v1beta1 0.15.3 gapic-google-cloud-vision-v1 0.90.3 gast 0.2.2 gcsfs 0.3.0 gensim 3.8.0 gevent 1.4.0 gitdb2 2.0.6 GitPython 3.0.3 glob2 0.7 gmpy2 2.0.8 google-api-core 1.26.2 google-api-python-client 1.11.0 google-auth 1.28.0 google-auth-httplib2 0.0.3 google-auth-oauthlib 0.4.1 google-cloud 0.34.0 google-cloud-bigquery 0.25.0 google-cloud-bigquery-datatransfer 0.1.1 google-cloud-bigtable 0.25.0 google-cloud-container 0.1.1 google-cloud-core 1.6.0 google-cloud-dataproc 0.5.0 google-cloud-datastore 1.4.0 google-cloud-dns 0.25.0 google-cloud-error-reporting 0.25.1 google-cloud-firestore 0.28.0 google-cloud-language 0.25.0 google-cloud-logging 1.4.0 google-cloud-monitoring 0.25.0 google-cloud-natural-language 0.24.0 google-cloud-pubsub 0.26.0 google-cloud-resource-manager 0.25.0 google-cloud-runtimeconfig 0.25.0 google-cloud-spanner 0.25.0 google-cloud-speech 0.26.0 google-cloud-storage 1.36.2 google-cloud-trace 0.17.0 google-cloud-translate 0.25.0 google-cloud-videointelligence 0.25.0 google-cloud-vision 0.25.1 google-colab 1.0.0 google-crc32c 1.0.0 google-gax 0.15.16 google-images-download 2.8.0 Google-Images-Search 1.1.2 google-pasta 0.1.8 google-resumable-media 1.2.0 googleapis-common-protos 1.51.0 graphviz 0.13.2 greenlet 0.4.15 grpc-google-iam-v1 0.11.4 grpcio 1.31.0 h5py 2.8.0 HeapDict 1.0.1 hmmlearn 0.2.4 html5lib 1.0.1 htmlmin 0.1.12 httplib2 0.19.0 huggingface-hub 0.0.2 hyperopt 0.2.2 idna 2.8 imageio 2.5.0 imageio-ffmpeg 0.4.3 imagesize 1.1.0 imbalanced-learn 0.6.2 importlib-metadata 0.23 imutils 0.5.3 inflect 4.1.0 install 1.3.1 ipykernel 4.6.1 ipython 5.5.0 ipython-genutils 0.2.0 ipython-sql 0.3.9 ipywidgets 7.5.1 isort 4.3.21 itsdangerous 1.1.0 jdcal 1.4.1 jedi 0.15.1 jeepney 0.4.1 Jinja2 2.10.1 jinja2-time 0.2.0 jiwer 2.2.0 jmespath 0.9.4 joblib 0.13.2 json5 0.8.5 jsonschema 3.0.2 jupyter 1.0.0 jupyter-aihub-deploy-extension 0.1 jupyter-client 5.3.3 jupyter-console 6.0.0 jupyter-contrib-core 0.3.3 jupyter-contrib-nbextensions 0.5.1 jupyter-core 4.5.0 jupyter-highlight-selected-word 0.2.0 jupyter-http-over-ws 0.0.6 jupyter-latex-envs 1.4.6 jupyter-nbextensions-configurator 0.4.1 jupyterlab 1.1.4 jupyterlab-git 0.8.1 jupyterlab-server 1.0.6 Keras 2.3.1 Keras-Applications 1.0.8 Keras-Preprocessing 1.1.0 keyring 18.0.0 kiwisolver 1.1.0 kubernetes 10.0.1 langdetect 1.0.8 lazy-object-proxy 1.4.2 libarchive-c 2.8 libcst 0.3.11 librosa 0.7.2 libsixel-python 0.4.0 lief 0.9.0 llvmlite 0.29.0 locket 0.2.0 lxml 4.4.1 Markdown 3.1.1 MarkupSafe 1.1.1 matplotlib 3.1.1 mccabe 0.6.1 missingno 0.4.2 mistune 0.8.4 mkl-fft 1.0.14 mkl-random 1.1.0 mkl-service 2.3.0 mock 3.0.5 more-itertools 7.2.0 moviepy 1.0.3 mpmath 1.1.0 msgpack 0.6.1 multidict 5.1.0 multipledispatch 0.6.0 multiprocess 0.70.11.1 murmurhash 1.0.2 mypy-extensions 0.4.3 mysql-connector-python 8.0.20 mysqlclient 2.0.1 nb-conda 2.2.1 nb-conda-kernels 2.2.2 nbconvert 5.6.0 nbdime 1.1.0 nbformat 4.4.0 nbpresent 3.0.2 networkx 2.3 nltk 3.4.5 nose 1.3.7 notebook 5.2.2 notebook-executor 0.1 numba 0.45.1 numexpr 2.7.0 numpy 1.17.4 numpydoc 0.9.1 oauth2client 3.0.0 oauthlib 3.1.0 olefile 0.46 opencv-python 4.1.1.26 openpyxl 3.0.0 opt-einsum 3.1.0 packaging 19.2 pandas 0.24.2 pandas-datareader 0.9.0 pandas-profiling 2.3.0 pandocfilters 1.4.2 papermill 1.2.0 parso 0.5.1 partd 1.0.0 path.py 12.0.1 pathlib2 2.3.5 patsy 0.5.1 pep8 1.7.1 pexpect 4.7.0 phik 0.9.8 pickleshare 0.7.5 Pillow 6.1.0 Pillow-SIMD 6.0.0.post0 pip 20.2.4 pkginfo 1.5.0.1 plac 0.9.6 plotly 4.1.1 pluggy 0.13.0 ply 3.8 portpicker 1.2.0 poyo 0.5.0 preshed 2.0.1 prettytable 0.7.2 proglog 0.1.9 prometheus-client 0.7.1 promise 2.2.1 prompt-toolkit 1.0.18 proto-google-cloud-datastore-v1 0.90.4 proto-google-cloud-error-reporting-v1beta1 0.15.3 proto-google-cloud-logging-v2 0.91.3 proto-google-cloud-pubsub-v1 0.15.4 proto-google-cloud-spanner-admin-database-v1 0.15.3 proto-google-cloud-spanner-admin-instance-v1 0.15.3 proto-google-cloud-spanner-v1 0.15.3 proto-google-cloud-speech-v1 0.15.3 proto-google-cloud-speech-v1beta1 0.15.3 proto-google-cloud-vision-v1 0.90.3 proto-plus 1.10.0 protobuf 3.13.0 psutil 5.6.3 pt-core-news-sm 2.1.0 ptyprocess 0.6.0 py 1.8.0 py4j 0.10.7 pyarrow 3.0.0 pyasn1 0.4.8 pyasn1-modules 0.2.7 pycodestyle 2.5.0 pycosat 0.6.3 pycparser 2.19 pycrypto 2.6.1 pycurl 7.43.0.3 pydot 1.4.1 pyfiglet 0.8.post1 pyflakes 2.1.1 Pygments 2.4.2 PyJWT 1.7.1 pylint 2.6.2 pymongo 3.10.1 pymssql 2.1.5 PyMySQL 0.9.3 pyodbc 4.0.27 pyOpenSSL 19.0.0 pyparsing 2.4.2 pyrsistent 0.15.4 PySocks 1.7.0 pyspark 2.4.6 pytest 5.2.0 pytest-arraydiff 0.3 pytest-astropy 0.5.0 pytest-doctestplus 0.4.0 pytest-openfiles 0.4.0 pytest-pylint 0.14.1 pytest-remotedata 0.3.2 python-dateutil 2.8.0 python-docx 0.8.10 python-Levenshtein 0.12.0 python-resize-image 1.1.19 pytorch-pretrained-bert 0.6.2 pytz 2019.2 PyWavelets 1.0.3 PyYAML 5.3.1 pyzmq 18.1.0 QtAwesome 0.6.0 qtconsole 4.5.5 QtPy 1.9.0 regex 2019.8.19 requests 2.25.1 requests-oauthlib 1.3.0 resampy 0.2.2 retrying 1.3.3 rope 0.14.0 rsa 4.0 ruamel-yaml 0.15.46 s3transfer 0.2.1 sacremoses 0.0.35 scikit-image 0.15.0 scikit-learn 0.22.1 scipy 1.3.1 seaborn 0.9.0 SecretStorage 3.1.1 selenium 3.141.0 Send2Trash 1.5.0 sentencepiece 0.1.91 setuptools 50.3.2 simplegeneric 0.8.1 singledispatch 3.4.0.3 six 1.15.0 smart-open 1.8.4 smmap2 2.0.5 snowballstemmer 1.9.1 sortedcollections 1.1.2 sortedcontainers 2.1.0 SoundFile 0.10.2 soupsieve 1.9.3 spacy 2.1.8 Sphinx 2.2.0 sphinxcontrib-applehelp 1.0.1 sphinxcontrib-devhelp 1.0.1 sphinxcontrib-htmlhelp 1.0.2 sphinxcontrib-jsmath 1.0.1 sphinxcontrib-qthelp 1.0.2 sphinxcontrib-serializinghtml 1.1.3 sphinxcontrib-websupport 1.1.2 spyder 3.3.6 spyder-kernels 0.5.2 SQLAlchemy 1.3.8 sqlparse 0.3.0 srsly 0.1.0 statsmodels 0.10.1 sympy 1.4 tables 3.4.4 tblib 1.4.0 tenacity 4.12.0 tensorboard 2.1.0 tensorflow 2.0.0 tensorflow-datasets 1.2.0 tensorflow-estimator 2.0.0 tensorflow-hub 0.7.0 tensorflow-metadata 0.14.0 termcolor 1.1.0 terminado 0.8.2 testpath 0.4.2 text2num 2.2.0 textwrap3 0.9.2 thinc 7.0.8 tokenizers 0.10.1 toml 0.10.2 toolz 0.10.0 torch 1.8.0 torchaudio 0.8.0 torchtext 0.6.0 torchvision 0.4.0a0+6b959ee tornado 4.5.3 tqdm 4.36.1 traitlets 4.3.2 transformers 4.4.0 typed-ast 1.4.0 typing-extensions 3.7.4.3 typing-inspect 0.6.0 unicodecsv 0.14.1 Unidecode 1.1.1 uritemplate 3.0.0 urllib3 1.24.2 virtualenv 16.7.5 wasabi 0.2.2 wcwidth 0.1.7 webencodings 0.5.1 websocket-client 0.56.0 Werkzeug 0.16.0 wheel 0.33.4 whichcraft 0.6.1 widgetsnbextension 3.5.1 word2number 1.1 wrapt 1.11.2 wurlitzer 1.0.3 xlrd 1.2.0 XlsxWriter 1.2.1 xlwt 1.3.0 xxhash 2.0.0 yarl 1.6.3 zict 1.0.0 zipp 0.6.0 zope.interface 5.1.0