search

elementary-data / elementary

The dbt-native data observability solution for data & analytics engineers. Monitor your data pipelines in minutes. Available as self-hosted or cloud service with premium features.
https://www.elementary-data.com/
Apache License 2.0
1.92k stars 165 forks source link

Filter send-report using multiple selectors #1686

Open stuartjd-lendable opened 2 months ago

stuartjd-lendable commented 2 months ago

Is your feature request related to a problem? Please describe. A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]

This is my understanding from some experimentation/testing but please let me know if there's a way to do this already.

Currently you can run the command edr send-report --select last_invocation which will filter the summary of the report to the last dbt execution. Alternatively you can run edr send-report --select tag:XYZ which will filter based on last 7 days (?) based on the tag selection.

However, running edr send-report --select last_invocation,tag:XYZ just sends the same thing as edr send-report --select last_invocation so there's no additional filtering going on.

Describe the solution you'd like A clear and concise description of what you want to happen.

It would be really useful to be able to apply selectors like in dbt selectors with the comma , meaning the intersection of the selectors i.e. edr send-report --select last_invocation,tag:XYZ would filter the summary to the last invocation and for tag XYZ. Same applies with filtering on last_invocation and the owners metadata etc.

Describe alternatives you've considered A clear and concise description of any alternative solutions or features you've considered.

Additional context Add any other context or screenshots about the feature request here.

Would you be willing to contribute this feature? Let us know if you're willing to contribute so that we'll be able to provide guidance.

I've had a quick look at the code and feel might be solved by updates to this set of if/elif conditions that rules out the possibility of multiple matches. I dont have time to familiarise enough with the codebase to know whether this is the solution or if there's a neater way of doing this. https://github.com/elementary-data/elementary/blob/85923fba4173d36763d9d3e71783d1cf8b1f7fcc/elementary/monitor/data_monitoring/selector_filter.py#L67

ofek1weiss commented 1 month ago

Hey @stuartjd-lendable , the report only supports the last_invocation, invocation_id and invocation_time selectors. after creating the report, you can filter the dashboard/results by tag inside of the report itself