search

data-mie / dbt-profiler

Macros for generating dbt model data profiles
Apache License 2.0
81 stars 33 forks source link

Using print_profile on dbt sources #55

Closed dweaver33 closed 2 years ago

dweaver33 commented 2 years ago

I can't seem to get the syntax correct for using the print_profile macro on command line with a dbt source.

I've tried the following:

dbt run-operation get_profile -t dev --args '{"relation":"source(my_source)"}'

And several other variations. Any ideas how this would work?

stumelius commented 2 years ago

Hi @dweaver33 ! In the example you gave you're using the get_profile macro instead of print_profile. Also, you're using the source macro which AFAIK doesn't work from the command line.

Try this instead:

dbt run-operation print_profile --args '{"relation_name": "my_table", "database": "my_database", "schema": "my_schema"}'

Let me know if that works!

dweaver33 commented 2 years ago

Ah, ok, that works.

One last question: Any way to put a WHERE clause on the profiler? Like if we wanted to only profile data that meets a certain criteria?

stumelius commented 2 years ago

@dweaver33 That's a good question and I haven't thought about that yet. Currently, there's no easy way to put a WHERE clause into the profiling query but adding it should be easy. Basically we'd need to add a where argument to the get_profile macro and all the other macros that use it. Would you be interested in contributing? :) I'd be happy to help and answer any questions you may have.

dweaver33 commented 2 years ago

Yeah, I'll give it a shot. I have a decent idea on where to put this, so let me give it a go and submit a PR. In the meantime, I'll open a new issue for tracking.