netlify[bot]
commented
7 months ago Deploy Preview for mellifluous-eclair-326dcd ready!
| Name | Link |
|---|---|
| Latest commit | 6ee1931ff8fb06dd835d56b5280c0aef332b75d9 |
| Latest deploy log | https://app.netlify.com/sites/mellifluous-eclair-326dcd/deploys/6626c4c596894f000832588c |
| Deploy Preview | https://deploy-preview-320--mellifluous-eclair-326dcd.netlify.app |
| Preview on mobile | Toggle QR Code...Use your smartphone camera to open QR code link. |
To edit notification comments on pull requests, go to your Netlify site configuration.
kacpermuda
DAG Code has been approved by @JDarDagran here
Problem
People face difficulty when setting up OpenLineage in Airflow. There is a lack of an easy way to verify if OpenLineage has been setup correctly and is able to send events, which can lead to difficulties in debugging.
Solution
Introducing a new DAG named "openlineage_preflight_check_dag" This DAG is designed to be executed by users who wish to verify their OpenLineage setup within Airflow. It serves as a diagnostic tool to ensure that OpenLineage has been integrated properly and is capable of sending events as expected.
The beauty of this solution lies in its compatibility with all versions and distributions of Airflow, making it a versatile tool for users across different environments.
This DAG takes care of both openlineage-airflow library as well as apache-airflow-providers-openlineage package
I have created placeholders for areas which require community inputs and task to be executed in case lineage backend is not Marquez
One-line summary:
openlineage_preflight_check_dagDAG which can be executed if user need to check if OpenLineage has been set up correctly or not.SPDX-License-Identifier: Apache-2.0\ Copyright 2018-2024 contributors to the OpenLineage project