apache / seatunnel

SeaTunnel is a next-generation super high-performance, distributed, massive data integration tool.
https://seatunnel.apache.org/
Apache License 2.0
8k stars 1.81k forks source link

[Umbrella][Spark][E2E] Enable spark e2e test cases of connectors #4356

Closed TyrantLucifer closed 1 year ago

TyrantLucifer commented 1 year ago

Code of Conduct

Search before asking

Describe the proposal

Currently, SeaTunnel has completed end-to-end testing for Flink versions 1.13-1.16 in terms of construction.

However, in the case of the Spark engine, some connectors' built-in dependencies conflict with certain dependencies of Spark, causing e2e testing of these connectors to fail in Spark.

In order to ensure the stability of the connectors, we need to solve these issues.

For more error details, you can refer to https://github.com/apache/incubator-seatunnel/pull/4286

Tips: This problem can be solved with maven-shade plugin.

Task list

Are you willing to submit PR?

MonsterChenzhuo commented 1 year ago

ES、Clickhouse、Amazon Dynamo DB、Neo4j please assign to me; thx。

FlechazoW commented 1 year ago

Http and jdbc, please.

TyrantLucifer commented 1 year ago

4534 is merged, this task has been done.