search

elementary-data / elementary

The dbt-native data observability solution for data & analytics engineers. Monitor your data pipelines in minutes. Available as self-hosted or cloud service with premium features.
https://www.elementary-data.com/
Apache License 2.0
1.8k stars 154 forks source link

Orchestrator support: Meltano #1160

Open menzenski opened 10 months ago

menzenski commented 10 months ago

Is your feature request related to a problem? Please describe.

https://docs.elementary-data.com/deployment-and-configuration/collect-job-data#cant-find-your-orchestrator-missing-info mentions that I should open an issue for an orchestrator that doesn't appear in the list.

We run dbt-core within a Meltano project and it would be very valuable to have job info from the Meltano invoker captured within elementary metadata.

Describe the solution you'd like

It would be great to have Meltano metadata captured in the same orchestrator metadata model as others:

Orchestrator name: meltano Job name: job_name (Meltano job_name) Job ID: n/a ? Job results URL: n/a ? The ID of a specific run execution: job_run_id (Meltano run_id ) Job run results URL: n/a ?

Describe alternatives you've considered not sure

Additional context

Would you be willing to contribute this feature? Yes!

elongl commented 5 months ago

Hi @menzenski, you can do it by filling the values in this macro. Here's our contribution guide. Let me know if you need any help.