Java: dataset namespace resolver feature[#2720](https://github.com/OpenLineage/OpenLineage/issues/2720)@pawel-big-lebowski Adds a dataset namespace resolving mechanism that resolves dataset namespaces based on the resolvers configured. The core mechanism is implemented in openlineage-java and can be used within the Flink and Spark integrations.
Spec: register GCP common job facet[#2740](https://github.com/OpenLineage/OpenLineage/issues/2740)@ngorchakova Registers the GCP job facet that contains common attributes that will improve the way lineage is parsed and displayed by the GCP platform. Based on the proposal, GCP Lineage would like to define facets that are expected from integrations. The list of support facets is not final and will be extended further by next PR.
Spark: remove spark-interfaces-scala module[#2781](https://github.com/OpenLineage/OpenLineage/issues/2781)@ddebowczyk92 Replaces the existing spark-interfaces-scala interfaces with new ones decoupled from the Scala binary version. Allows for improved integration in environments where one cannot guarantee the same version of openlineage-java.
Spark, Flink: add OracleJdbcExtractor[#2824](https://github.com/OpenLineage/OpenLineage/issues/2824)@dolfinus Handles simple Oracle JDBC URLs, like oracle:thin:@//host:port/serviceName and oracle:thin@host:port:sid, and converts each to a dataset with namespace oracle://host:port and name sid.schema.table or serviceName.schema.table.
Spec: add example for change in [#2756](https://github.com/OpenLineage/OpenLineage/issues/2756)[#2801](https://github.com/OpenLineage/OpenLineage/issues/2801)@Sheeri Updates the customLineage facet test for the new syntax created in [#2756](https://github.com/OpenLineage/OpenLineage/issues/2756).
Changed
Spark: fallback to spark.sql.warehouse.dir as table namespace[#2767](https://github.com/OpenLineage/OpenLineage/issues/2767)@dolfinus In cases when a metastore is not used, falls back to spark.sql.warehouse.dir or hive.metastore.warehouse.dir as table namespace, instead of duplicating the table's location.
Spark: convert scheme and authority to lowercase in JdbcLocation[#2831](https://github.com/OpenLineage/OpenLineage/issues/2831)@dolfinus Converts valid JDBC URL scheme and authority to lowercase, leaving intact instance/database name, as different databases have different default case and case-sensitivity rules.
Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.
Dependabot commands and options
You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
- `@dependabot show ignore conditions` will show all of the ignore conditions of the specified dependency
- `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
dependabot[bot]
commented
4 days ago
Bumps io.openlineage:openlineage-java from 1.16.0 to 1.18.0.
Release notes
Sourced from io.openlineage:openlineage-java's releases.
Changelog
Sourced from io.openlineage:openlineage-java's changelog.
... (truncated)
Commits
546cef5Prepare for release 1.18.0813737aPrepare next development version 1.19.0-SNAPSHOT2ecd826Prepare for release 1.18.03141fbaUpdate changelog (#2846)00a7e9cspec: JobTypeJobFacet description correctly describes possible values in fiel...dc28117iceberg support for Spark 3.5.1 (#2838)338876esql: remove debug println (#2844)035cfe3fix configurable integration test (#2843)5631150[SPARK] Set spark-extension-entrypoint version to 1.0.0 (#2837)fcc6bdbFix: handle dashes in hostname for JdbcExtractors (#2830)Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting
@dependabot rebase.Dependabot commands and options
You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot merge` will merge this PR after your CI passes on it - `@dependabot squash and merge` will squash and merge this PR after your CI passes on it - `@dependabot cancel merge` will cancel a previously requested merge and block automerging - `@dependabot reopen` will reopen this PR if it is closed - `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually - `@dependabot show