data-mie / dbt-cloud-cli

dbt Cloud command line interface (CLI)
Apache License 2.0
71 stars 9 forks source link

Allow `dbt-cloud metadata query` to accept the query as stdin/inline string #55

Closed b-per closed 2 years ago

b-per commented 2 years ago

First of all, thanks a lot for the work here, this is amazing.

I have been looking at dbt-cloud metadata query and was expecting to be able to run a query like:

dbt-cloud metadata query '{
  model(jobId: xxxxx, uniqueId:"xxxx") {
    name
    uniqueId
    type
    jobId
  }
}'

but it returned Error: Got unexpected extra argument ({ ........})

I was able to run the query by saving it in a file and using -f file_name though.

Is there a way to run a metadata query inline or is the only way with -f ? My use case was to get all the jobs with dbt-cloud job list and pipe this list with xargs to run a metadata query for each job.

If this is not possible today, would you be OK with a contribution to make it possible?

stumelius commented 2 years ago

@b-per The -f argument takes a piped input by default, i.e. this should work:

echo '{
  model(jobId: xxxxx, uniqueId:"xxxx") {
    name
    uniqueId
    type
    jobId
  }
}' | dbt-cloud metadata query

Let me know if this fits your use case :)

b-per commented 2 years ago

Thanks for the answer, it worked! Not sure if is worth adding an example without the -f in the README. (I can do it if you agree that another example could help).

For my use case (listing all models run by all jobs), the following pipes of commands gave me the output I was after:

dbt-cloud job list | jq '.data[].id' | xargs -I % sh -c 'echo "query {
  models(jobId: %){
    runId
    uniqueId
  }
}" | dbt-cloud metadata query'
stumelius commented 2 years ago

@b-per I saw you made a PR for this, thanks! I've been meaning to get to this all week but haven't had the chance yet so I really appreciate your help :)