Google Cloud Storage Bucket component

pmuens commented 6 years ago

Description

Implement a google-cloud-storage-bucket component. Specifically, this component will be used to setup and manage a google cloud storage bucket. This component is considered a "low level" component should adhere as close as possible the Google's API for inserting a new bucket.

API

The component inputTypes should match the parameters which can be defined in the APIs insert operation

The components' deploy and remove methods should behave as idempotent operations.

For deploy, if a bucket with the name already exists, the component should attempt to update the bucket with the given inputs. If the bucket cannot be updated, an error should be thrown.

For remove, if a bucket with the given name does not exist an error should not be thrown, instead (if an error occurs, it should be swallowed and execution should be allowed to proceed.)

inputTypes

`name` - `string`

The name of the bucket. Must adhere to the bucket naming conventions

`project` - `string`

A valid API project identifier.

`predefinedAcl` - `string (optional)`

Apply a predefined set of access controls to this bucket. Acceptable values are:

"authenticatedRead": Project team owners get OWNER access, and allAuthenticatedUsers get READER access.
"private": Project team owners get OWNER access.
"projectPrivate": Project team members get access according to their roles.
"publicRead": Project team owners get OWNER access, and allUsers get READER access.
"publicReadWrite": Project team owners get OWNER access, and allUsers get WRITER access.

`predefinedDefaultObjectAcl` - `string(optional)`

Apply a predefined set of default object access controls to this bucket. Acceptable values are:

"authenticatedRead": Object owner gets OWNER access, and allAuthenticatedUsers get READER access.
"bucketOwnerFullControl": Object owner gets OWNER access, and project team owners get OWNER access.
"bucketOwnerRead": Object owner gets OWNER access, and project team owners get READER access.
"private": Object owner gets OWNER access.
"projectPrivate": Object owner gets OWNER access, and project team members get access according to their roles.
"publicRead": Object owner gets OWNER access, and allUsers get READER access.

`projection` - `string(optional)`

Set of properties to return. Defaults to noAcl, unless the bucket resource specifies acl or defaultObjectAclproperties, when it defaults to full. Acceptable values are:

"full": Include all properties.
"noAcl": Omit owner, acl and defaultObjectAclproperties.

`userProject` - `string (optional)`

The project to be billed for this request.

`acl` - `array (optional)`

Access controls on the bucket, containing one or more bucketAccessControls Resources.

`billing` - `object (optional)`

The bucket's billing configuration.

`billing.requesterPays` - `boolean (optional)`

When set to true, Requester Pays is enabled for this bucket.

`cors` - `array (optional)`

The bucket's Cross-Origin Resource Sharing (CORS) configuration.

`cors[].maxAgeSeconds` - `integer (optional)`

The value, in seconds, to return in the Access-Control-Max-Age header used in preflight responses.

`cors[].method` - `array<string> (optional)`

The list of HTTP methods on which to include CORS response headers, (GET, OPTIONS, POST, etc) Note: "*" is permitted in the list of methods, and means "any method".

`cors[].origin` - `array<string> (optional)`

The list of Origins eligible to receive CORS response headers. Note: "*" is permitted in the list of origins, and means "any Origin".

`cors[].responseHeader` - `array<string> (optional)`

The list of HTTP headers other than the simple response headers to give permission for the user-agent to share across domains.

`defaultObjectAcl` - `array<Object> (optional)`

Default access controls to apply to new objects when no ACL is provided.

`defaultObjectAcl[].entity` - `string (optional)`

The entity holding the permission, in one of the following forms:user-userIduser-emailgroup-groupIdgroup-emaildomain-domainproject-team-projectIdallUsersallAuthenticatedUsersExamples:The user liz@example.com would be user-liz@example.com.The group example@googlegroups.com would be group-example@googlegroups.com.To refer to all members of the G Suite for Business domain example.com, the entity would be domain-example.com.

`defaultObjectAcl[].role` - `string (optional)`

The access permission for the entity. Acceptable values are:

"OWNER"
"READER"

`encryption` - `object (optional)`

Encryption configuration for a bucket.

`encryption.defaultKmsKeyName (optional)` - `string`

A Cloud KMS key that will be used to encrypt objects inserted into this bucket, if no encryption method is specified.

`labels` - `object (optional)`

User-provided labels, in key/value pairs.

`labels.(key)` - `string (optional)`

An individual label entry.

`lifecycle` - `object (optional)`

The bucket's lifecycle configuration. See lifecycle management for more information.

`location` - `string (optional)`

The location of the bucket. Object data for objects in the bucket resides in physical storage within this region. Defaults to US. See the developer's guide for the authoritative list.

`logging` - `object (optional)`

The bucket's logging configuration, which defines the destination bucket and optional name prefix for the current bucket's logs.

`logging.logBucket` - `string (optional)`

The destination bucket where the current bucket's logs should be placed.

`logging.logObjectPrefix` - `string (optional)`

A prefix for log object names.

`storageClass` - `string (optional)`

The bucket's default storage class, used whenever no storageClass is specified for a newly-created object. This defines how objects in the bucket are stored and determines the SLA and the cost of storage. Acceptable Values

MULTI_REGIONAL
REGIONAL
STANDARD
NEARLINE
COLDLINE
DURABLE_REDUCED_AVAILABILITY If this value is not specified when the bucket is created, it will default to STANDARD. For more information, see storage classes.

`versioning` - `object (optional)`

The bucket's versioning configuration.

`versioning.enabled` - `boolean (optional)`

While set to true, versioning is fully enabled for this bucket.

`website` - `object (optional)`

The bucket's website configuration, controlling how the service behaves when accessing bucket contents as a web site. See the Static Website Examples for more information.

`website.mainPageSuffix` - `string (optional)`

If the requested object path is missing, the service will ensure the path has a trailing '/', append this suffix, and attempt to retrieve the resulting object. This allows the creation of index.html objects to represent directory pages.

`website.notFoundPage` - `string (optional)`

If the requested object path is missing, and any mainPageSuffix object is missing, if applicable, the service will return the named object from this bucket as the content for a 404 Not Found result.

outputTypes

`acl` - `array`

Access controls on the bucket, containing one or more bucketAccessControls Resources.

`billing` - `object`

The bucket's billing configuration.

`billing.requesterPays` - `boolean`

When set to true, Requester Pays is enabled for this bucket.

`cors` - `array<Object>`

The bucket's Cross-Origin Resource Sharing (CORS) configuration.

`cors[].maxAgeSeconds` - `integer`

The value, in seconds, to return in the Access-Control-Max-Age header used in preflight responses.

`cors[].method` - `array<string>`

The list of HTTP methods on which to include CORS response headers, (GET, OPTIONS, POST, etc) Note: "*" is permitted in the list of methods, and means "any method".

`cors[].origin` - `array<string>`

The list of Origins eligible to receive CORS response headers. Note: "*" is permitted in the list of origins, and means "any Origin".

`cors[].responseHeader` - `array<string>`

The list of HTTP headers other than the simple response headers to give permission for the user-agent to share across domains.

`defaultObjectAcl` - `array<Object>`

Default access controls to apply to new objects when no ACL is provided.

`defaultObjectAcl[].bucket` - `string`

The name of the bucket.

`defaultObjectAcl[].domain` - `string`

The domain associated with the entity, if any.

`defaultObjectAcl[].email` - `string`

The email address associated with the entity, if any.

`defaultObjectAcl[].entity` - `string`

The entity holding the permission, in one of the following forms:user-userIduser-emailgroup-groupIdgroup-emaildomain-domainproject-team-projectIdallUsersallAuthenticatedUsersExamples:The user liz@example.com would be user-liz@example.com.The group example@googlegroups.com would be group-example@googlegroups.com.To refer to all members of the G Suite for Business domain example.com, the entity would be domain-example.com.

`defaultObjectAcl[].entityId` - `string`

The ID for the entity, if any.

`defaultObjectAcl[].etag` - `string`

HTTP 1.1 Entity tag for the access-control entry.

`defaultObjectAcl[].generation` - `long`

The content generation of the object, if applied to an object.

`defaultObjectAcl[].id` - `string`

The ID of the access-control entry.

`defaultObjectAcl[].kind` - `string`

The kind of item this is. For object access control entries, this is always storage#objectAccessControl.

`defaultObjectAcl[].object` - `string`

The name of the object, if applied to an object.

`defaultObjectAcl[].projectTeam` - `object`

The project team associated with the entity, if any.

`defaultObjectAcl[].projectTeam.projectNumber` - `string`

The project number.

`defaultObjectAcl[].projectTeam.team` - `string`

The team. Acceptable values are:"editors""owners""viewers"

`defaultObjectAcl[].role` - `string`

The access permission for the entity. Acceptable values are:

"OWNER"
"READER"

`defaultObjectAcl[].selfLink` - `string`

The link to this access-control entry.

`encryption` - `object`

Encryption configuration for a bucket.

`encryption.defaultKmsKeyName (optional)` - `string`

A Cloud KMS key that will be used to encrypt objects inserted into this bucket, if no encryption method is specified

`etag` - `string`

HTTP 1.1 Entity tag for the bucket.

`id` - `string`

The ID of the bucket. For buckets, the id and name properties are the same.

`kind` - `string`

The kind of item this is. For buckets, this is always storage#bucket.

`labels` - `object`

User-provided labels, in key/value pairs.

`labels.(key)` - `string`

An individual label entry.

`lifecycle` - `object (optional)`

The bucket's lifecycle configuration. See lifecycle management for more information.

`lifecycle.rule` - `array<Object>`

A lifecycle management rule, which is made of an action to take and the condition(s) under which the action will be taken.

`lifecycle.rule[].action` - `object`

The action to take.

`lifecycle.rule[].action.storageClass` - `string`

Target storage class. Required iff the type of the action is SetStorageClass.

`lifecycle.rule[].action.type` - `string`

Type of the action. Currently, only Delete and SetStorageClass are supported. Acceptable values are:"Delete""SetStorageClass"

`lifecycle.rule[].condition` - `object`

The condition(s) under which the action will be taken.

`lifecycle.rule[].condition.age` - `integer`

Age of an object (in days). This condition is satisfied when an object reaches the specified age.

`lifecycle.rule[].condition.createdBefore` - `date`

A date in RFC 3339 format with only the date part (for instance, "2013-01-15"). This condition is satisfied when an object is created before midnight of the specified date in UTC.

`lifecycle.rule[].condition.isLive` - `boolean`

Relevant only for versioned objects. If the value is true, this condition matches live objects; if the value is false, it matches archived objects.

`lifecycle.rule[].condition.matchesStorageClass` - `array<string>`

Objects having any of the storage classes specified by this condition will be matched. Values include

MULTI_REGIONAL
REGIONAL
NEARLINE
COLDLINE
STANDARD
DURABLE_REDUCED_AVAILABILITY.
lifecycle.rule[].condition.numNewerVersions - integer

Relevant only for versioned objects. If the value is N, this condition is satisfied when there are at least N versions (including the live version) newer than this version of the object.

`location` - `string`

The location of the bucket. Object data for objects in the bucket resides in physical storage within this region. Defaults to US. See the developer's guide for the authoritative list.

`logging` - `object`

The bucket's logging configuration, which defines the destination bucket and optional name prefix for the current bucket's logs.

`logging.logBucket` - `string`

The destination bucket where the current bucket's logs should be placed.

`logging.logObjectPrefix` - `string`

A prefix for log object names.

`metageneration` - `long`

The metadata generation of this bucket.

`name` - `string`

The name of the bucket.

`owner` - `object`

The owner of the bucket. This is always the project team's owner group.

`owner.entity` - `string`

The entity, in the form project-owner-projectId.

`owner.entityId` - `string`

The ID for the entity.

`projectNumber` - `unsigned long`

The project number of the project the bucket belongs to.

`selfLink` - `string`

The URI of this bucket.

`storageClass` - `string (optional)`

The bucket's default storage class, used whenever no storageClass is specified for a newly-created object. This defines how objects in the bucket are stored and determines the SLA and the cost of storage. Acceptable Values

MULTI_REGIONAL
REGIONAL
STANDARD
NEARLINE
COLDLINE
DURABLE_REDUCED_AVAILABILITY If this value is not specified when the bucket is created, it will default to STANDARD. For more information, see storage classes.

`timeCreated` - `datetime`

The creation time of the bucket in RFC 3339 format.

`updated` - `datetime`

The modification time of the bucket in RFC 3339 format.

`versioning` - `object (optional)`

The bucket's versioning configuration.

`versioning.enabled` - `boolean (optional)`

While set to true, versioning is fully enabled for this bucket.

`website` - `object (optional)`

The bucket's website configuration, controlling how the service behaves when accessing bucket contents as a web site. See the Static Website Examples for more information.

`website.mainPageSuffix` - `string (optional)`

If the requested object path is missing, the service will ensure the path has a trailing '/', append this suffix, and attempt to retrieve the resulting object. This allows the creation of index.html objects to represent directory pages.

`website.notFoundPage` - `string (optional)`

If the requested object path is missing, and any mainPageSuffix object is missing, if applicable, the service will return the named object from this bucket as the content for a 404 Not Found result.

minimal `serverless.yml` example of using this component

type: my-application
version: 0.0.2

components:
  myGoogleCloudStorageBucket:
    type: google-cloud-storage-bucket
    inputs:
      name: my-unique-bucket
      project: my-project

Implementation

[ ] tests written
[ ] inputs and outputs documented
[ ] example of using component added to examples folder

grayside commented 6 years ago

Is the goal of google-cloud-storage to encompass buckets, file uploads, ACLs, and so on?

grayside commented 6 years ago

I'm starting to explore this, starting with a component I've named google-storage-bucket on the notion that google-cloud-storage is likely to be a component family.

brianneisler commented 6 years ago

@grayside I've updated the description here to match what the intended implementation of this component should be. Most of this was copy and paste from the google storage bucket docs. Let me know if anything isn't clear or if you have questions or a different idea for implementation.

grayside commented 5 years ago

I'm working on an "MVP" version of this with a small number of simple options.

eahefnawy commented 5 years ago

We'd still love to get this in the new implementation 😊

serverless / components