googleapis / python-storage

Apache License 2.0
450 stars 154 forks source link

Python Client for Google Cloud Storage

|stable| |pypi| |versions|

Google Cloud Storage_ is a managed service for storing unstructured data. Cloud Storage allows world-wide storage and retrieval of any amount of data at any time. You can use Cloud Storage for a range of scenarios including serving website content, storing data for archival and disaster recovery, or distributing large data objects to users via direct download.

A comprehensive list of changes in each version may be found in the CHANGELOG_.

Certain control plane and long-running operations for Cloud Storage (including Folder and Managed Folder operations) are supported via the Storage Control Client. The Storage Control API creates one space to perform metadata-specific, control plane, and long-running operations apart from the Storage API.

Read more about the client libraries for Cloud APIs, including the older Google APIs Client Libraries, in Client Libraries Explained_.

.. |stable| image:: https://img.shields.io/badge/support-stable-gold.svg :target: https://github.com/googleapis/google-cloud-python/blob/main/README.rst#stability-levels .. |pypi| image:: https://img.shields.io/pypi/v/google-cloud-storage.svg :target: https://pypi.org/project/google-cloud-storage/ .. |versions| image:: https://img.shields.io/pypi/pyversions/google-cloud-storage.svg :target: https://pypi.org/project/google-cloud-storage/ .. _Google Cloud Storage: https://cloud.google.com/storage .. _Client Library Documentation: https://cloud.google.com/python/docs/reference/storage/latest .. _Product Documentation: https://cloud.google.com/storage .. _CHANGELOG: https://github.com/googleapis/python-storage/blob/main/CHANGELOG.md .. _github.com/googleapis/python-storage: https://github.com/googleapis/python-storage .. _Storage Control Client: https://cloud.google.com/python/docs/reference/google-cloud-storage-control/latest .. _Storage Control API: https://cloud.google.com/storage/docs/reference/rpc/google.storage.control.v2 .. _Client Libraries Explained: https://cloud.google.com/apis/docs/client-libraries-explained

Quick Start

In order to use this library, you first need to go through the following steps. A step-by-step guide may also be found in Get Started with Client Libraries_.

  1. Select or create a Cloud Platform project._
  2. Enable billing for your project._
  3. Enable the Google Cloud Storage API._
  4. Setup Authentication._

.. _Get Started with Client Libraries: https://cloud.google.com/storage/docs/reference/libraries#client-libraries-install-python .. _Select or create a Cloud Platform project.: https://console.cloud.google.com/project .. _Enable billing for your project.: https://cloud.google.com/billing/docs/how-to/modify-project#enable_billing_for_a_project .. _Enable the Google Cloud Storage API.: https://console.cloud.google.com/flows/enableapi?apiid=storage-api.googleapis.com .. _Setup Authentication.: https://cloud.google.com/docs/authentication/client-libraries

Installation


Install this library in a virtual environment using `venv`_. `venv`_ is a tool that
creates isolated Python environments. These isolated environments can have separate
versions of Python packages, which allows you to isolate one project's dependencies
from the dependencies of other projects.

With `venv`_, it's possible to install this library without needing system
install permissions, and without clashing with the installed system
dependencies.

.. _`venv`: https://docs.python.org/3/library/venv.html

Code samples and snippets

Code samples and snippets live in the samples/_ folder.

.. _samples/: https://github.com/googleapis/python-storage/tree/main/samples

Supported Python Versions ^^^^^^^^^^^^^^^^^^^^^^^^^ Our client libraries are compatible with all current active and maintenance versions of Python.

Python >= 3.7

.. _active: https://devguide.python.org/devcycle/#in-development-main-branch .. _maintenance: https://devguide.python.org/devcycle/#maintenance-branches

Unsupported Python Versions ^^^^^^^^^^^^^^^^^^^^^^^^^^^ Python <= 3.6

If you are using an end-of-life_ version of Python, we recommend that you update as soon as possible to an actively supported version.

.. _end-of-life: https://devguide.python.org/devcycle/#end-of-life-branches

Mac/Linux ^^^^^^^^^

.. code-block:: console

python3 -m venv <your-env>
source <your-env>/bin/activate
pip install google-cloud-storage

Windows ^^^^^^^

.. code-block:: console

py -m venv <your-env>
.\<your-env>\Scripts\activate
pip install google-cloud-storage

Tracing With OpenTelemetry


This is a PREVIEW FEATURE: Coverage and functionality are still in development and subject to change.

This library can be configured to use `OpenTelemetry`_ to generate traces on calls to Google Cloud Storage.
For information on the benefits and utility of tracing, read the `Cloud Trace Overview <https://cloud.google.com/trace/docs/overview>`_.

To enable OpenTelemetry tracing in the Cloud Storage client, first install OpenTelemetry:

.. code-block:: console

    pip install google-cloud-storage[tracing]

Set the ``ENABLE_GCS_PYTHON_CLIENT_OTEL_TRACES`` environment variable to selectively opt-in tracing for the Cloud Storage client:

.. code-block:: console

    export ENABLE_GCS_PYTHON_CLIENT_OTEL_TRACES=True

You will also need to tell OpenTelemetry which exporter to use. An example to export traces to Google Cloud Trace can be found below.

.. code-block:: console

    # Install the Google Cloud Trace exporter and propagator, however you can use any exporter of your choice.
    pip install opentelemetry-exporter-gcp-trace opentelemetry-propagator-gcp

    # [Optional] Install the OpenTelemetry Requests Instrumentation to trace the underlying HTTP requests.
    pip install opentelemetry-instrumentation-requests

.. code-block:: python

    from opentelemetry import trace
    from opentelemetry.sdk.trace import TracerProvider
    from opentelemetry.sdk.trace.export import BatchSpanProcessor
    from opentelemetry.exporter.cloud_trace import CloudTraceSpanExporter

    tracer_provider = TracerProvider()
    tracer_provider.add_span_processor(BatchSpanProcessor(CloudTraceSpanExporter()))
    trace.set_tracer_provider(tracer_provider)

    # Optional yet recommended to instrument the requests HTTP library
    from opentelemetry.instrumentation.requests import RequestsInstrumentor
    RequestsInstrumentor().instrument(tracer_provider=tracer_provider)

In this example, tracing data will be published to the `Google Cloud Trace`_ console.
Tracing is most effective when many libraries are instrumented to provide insight over the entire lifespan of a request.
For a list of libraries that can be instrumented, refer to the `OpenTelemetry Registry`_.

.. _OpenTelemetry: https://opentelemetry.io
.. _OpenTelemetry Registry: https://opentelemetry.io/ecosystem/registry
.. _Google Cloud Trace: https://cloud.google.com/trace

Next Steps

.. _Google Cloud Storage Product documentation: https://cloud.google.com/storage .. _README: https://github.com/googleapis/google-cloud-python/blob/main/README.rst