search

dataform-co / dataform

Dataform is a framework for managing SQL based data operations in BigQuery
https://cloud.google.com/dataform/docs
Apache License 2.0
848 stars 160 forks source link

Implement tags for actions #229

Closed lewish closed 5 years ago

lewish commented 5 years ago

All action types, datasets, operations, and assertions - should carry a list of tags that can be added to that action type.

Tags are simply a list of string labels that can be added to an action. These will be used for specifying schedules in the future.

For example, with the new configuration syntax:

config {
  tags: ["tag1", "tag2"]
}
BenBirt commented 5 years ago

This would involve:

  1. Adding a new tags property on the ISqlxConfig interface, giving it a default value of [] at https://github.com/dataform-co/dataform/blob/master/core/compilers.ts#L92, and passing it through to the dataset/operation/assertion here: https://github.com/dataform-co/dataform/blob/master/core/session.ts#L56
  2. Defining a new field on the relevant protobuf messages (Table, Operation, and Assertion) here: https://github.com/dataform-co/dataform/blob/master/protos/core.proto, and populating it with the result of (1)
  3. For bonus points, extending the JS API to allow users to specify this, by defining new tags() methods on Table, Operation, and Assertion (all in the /core/ directory).
faiyaz26 commented 5 years ago

One PR on this issue: https://github.com/dataform-co/dataform/pull/304

BenBirt commented 5 years ago

fixed with https://github.com/dataform-co/dataform/pull/304

kennethkenneth commented 5 years ago

Please reopen this issue or open a new one to include code that can resove what graph actions to run. Ping @BenBirt