search

apache / seatunnel

SeaTunnel is a next-generation super high-performance, distributed, massive data integration tool.
https://seatunnel.apache.org/
Apache License 2.0
8.04k stars 1.82k forks source link

[Feature][Transform] Support JOIN operations in transform and create complex graph ETL #7266

Open maerman13 opened 3 months ago

maerman13 commented 3 months ago

Search before asking

Description

Hello!

  1. Do you have plan to support JOIN operations for transform? For example, read data from multiple sources then join it and write result to one sink?
  2. Do you have plan to support complex graph ETL like Apache Airflow?

Usage Scenario

  1. Read data from source $1 (for example, Oracle)
  2. Read data from source $2 (for example, PG)
  3. Join data from source 1 and source 2 in transform
  4. Write to sink

Related issues

No response

Are you willing to submit a PR?

Code of Conduct

hailin0 commented 3 months ago

cc @rewerma

github-actions[bot] commented 2 months ago

This issue has been automatically marked as stale because it has not had recent activity for 30 days. It will be closed in next 7 days if no further activity occurs.

hexnn commented 2 months ago

Usage Scenario Read data from source $1 (for example, Oracle) Read data from source $2 (for example, PG) Join data from source 1 and source 2 in transform Write to sink

@maerman13 Try this：https://github.com/hexnn/stark