treasure-data / digdag

Workload Automation System
https://www.digdag.io/
Apache License 2.0
1.3k stars 221 forks source link

Implement google sheet export #29

Closed danielnorberg closed 8 years ago

danielnorberg commented 8 years ago

E.g.

+foo:
  td>: queries/foo.sql
  google_sheet: domain.com/foo/bar

Would perform the equivalent of

td query \
  --result 'gspreadsheet://domain.com/foo/bar' \
  -w -d ${td.database} \
  -q queries/foo.sql
frsyuki commented 8 years ago

Using "google_sheet" as a top level key is probably confusing both for code and users. And TD's REST API wants to obsolete URL-based result parameters (implemented in v3). TD wants to use json-based description (v4).

So, td> operator wants to do this in the future:

+foo:
  td>: queries/foo.sql
  result:
    type: bigquery
    project: myproject
    dataset: abc
    table: foo
    bad_records: 10

but will be like this for now:

+foo:
  td>: queries/foo.sql
  result_url: gspreadsheet://domain.com/foo/bar
danielnorberg commented 8 years ago

:+1:

frsyuki commented 8 years ago

And as you can find at the document, result_url option is already available (since digdag v0.2.1).