search

apache / seatunnel

SeaTunnel is a next-generation super high-performance, distributed, massive data integration tool.
https://seatunnel.apache.org/
Apache License 2.0
7.86k stars 1.77k forks source link

[Umbrella][Connector] New SeaTunnel API Connectors #1946

Closed Hisoka-X closed 1 year ago

Hisoka-X commented 2 years ago

#

Please Move To https://github.com/apache/incubator-seatunnel/issues/3018

#

No Connector priority difficulty Source/Sink Contributer Status Issue/PR
1 Console high low Sink @ruanwenjun Done #1864
2 Fake high low Source @ruanwenjun Done #1864
3 Doris high high Source @2013650523 #2536
4 Doris high high Sink @hk-lrzy #2586
5 Druid high middle Source @guanboo #2937
6 Druid high middle Sink @guanboo #2937
7 ElasticSearch high high Source @iture123 #2821
8 ElasticSearch high high Sink @iture123 Done #2330
9 ClickHouse high high Source @Hisoka-X Done #2051
10 ClickHouse high high Sink @Hisoka-X Done #2051
11 JDBC high high Source @ic4y Done #2048
12 JDBC high high Sink @ic4y Done #2048
13 FeiShu low low Source @TyrantLucifer
14 FeiShu low low Sink @TyrantLucifer Done
15 File(Local) high middle Source @TyrantLucifer Done
16 File(Local) high middle Sink @EricJoy2048 Done #2117
17 File(S3) high middle Source @TyrantLucifer
18 File(S3) high middle Sink @TyrantLucifer
19 File(HDFS) high high Source @TyrantLucifer Done
20 File(HDFS) high high Sink @EricJoy2048 Done
21 File(OSS) high middle Source @TyrantLucifer Done
22 File(OSS) high middle Sink @TyrantLucifer Done
23 Hudi high high Source @Emor-nj Done #2147
24 Hudi high high Sink @Emor-nj
25 Iceberg high middle Source @hailin0 Done #2615
26 Iceberg high middle Sink @s7monk
27 Kafka high middle Source @Hisoka-X Done
28 Kafka high middle Sink @ruanwenjun Done
29 Kudu high high Source @2013650523 Done #2254
30 Kudu high high Sink @2013650523 Done #2254
31 MongoDB high middle Source @wuchunfu Done
32 MongoDB high middle Sink @wuchunfu Done
33 Neo4j middle middle Source @getChan #2777
34 Neo4j middle middle Sink @getChan Done #2434
35 Phoenix middle middle Source @531651225 Done #2499
36 Phoenix middle middle Sink @531651225 Done #2499
37 Redis high middle Source @TyrantLucifer Done
38 Redis high middle Sink @TyrantLucifer Done
39 Socket high low Source @zhuangchong Done #1999
40 Socket high low Sink @531651225 Done #2549
41 [JDBC]Tidb high low Source @xbkaishui Abandonment
42 [JDBC]Tidb high low Sink @xbkaishui Abandonment
43 Webhook high middle Sink @TyrantLucifer Done #2348
44 InfluxDB middle middle Source @531651225 #2697
45 InfluxDB middle middle Sink @Jellal-HT
46 Pulsar high middle Source @ashulin Done #1984
47 Pulsar high middle Sink @FlechazoW
48 Email middle low Sink @2013650523 Done #2304
49 Assert low low Sink @lhyundeadsoul Done
50 Redshift high high Source
51 Facebook Marketing middle middle Source
52 HubSpot middle middle Source
53 Instagram middle middle Source
54 Bing ADs middle middle Source
55 Google Analytics middle middle Source @MRYOG
56 Intercom middle middle Source
57 Zendesk middle middle Source
58 TikTok Marketing middle middle Source
59 Salesforce middle middle Source
60 LinkedIn Ads middle middle Source
61 Stripe middle middle Source
62 Sentry middle middle Sink @Saintyang done #2244
63 DingTalk low low Source @MRYOG #2757
63 DingTalk low low Sink @MRYOG Done #2257
64 IoTDB high high Source @CalvinKirs Done
65 IoTDB high high Sink @hailin0 Done
66 TD-engine middle middle Source @lhyundeadsoul #2707
67 TD-engine middle middle Sink @lhyundeadsoul
68 HBase high high Source @zhuzhengjun01
69 HBase high high Sink @zhuzhengjun01
70 Hive high high Source @CalvinKirs Done
71 Hive high high Sink @EricJoy2048 Done
72 HTTP high middle Source @zhuangchong Done #2012
72 StarRocks high high Sink @tonyDong-code
73 StarRocks high high Source @wangw9420
74 ADB PostgreSQL high low Sink @etcZYP
75 Greenplum high high Source @hailin0 Done #2429
76 Greenplum high high Sink @hailin0 Done #2429
77 OceanBase middle high Sink @silenceland
78 OceanBase middle high Source @silenceland
79 DB2 high high Sink @laglangyue #2410
80 DB2 high high Source @laglangyue #2410
81 BigSource middle middle Source
82 Github low low Source @MonsterChenzhuo
83 Enterprise WeChat low low Sink @531651225 Done #2412
84 Slack middle low Sink @Charlie17Li
85 Databricks Lakehouse high high Sink
86 Snowflake middle high Sink
87 Snowflake middle high Source
88 [Jdbc]Sql-Server middle low Sink @liugddx Done #2646
89 [Jdbc]Sql-Server middle low Source @liugddx Done #2646
90 [Jdbc]Oracle high low Sink @liugddx #2550
91 [Jdbc]Oracle high low Source @liugddx #2550
92 [JDBC]Rds high low Sink @s7monk #2829
93 [JDBC]Rds high low Source @s7monk #2829
94 [JDBC]SqlLite middle low Sink @maruko-code
95 [JDBC]SqlLite middle low Source @Caribbeanz
96 [JDBC]DM(达梦) middle low Source @laglangyue done #2377
97 [JDBC]DM(达梦) middle low Sink @laglangyue done #2377
98 Cassandra middle middle Sink @bigdataf
99 Cassandra middle middle Source @bigdataf
100 [File]excel middle low Sink @Bingz2 #2585
101 [File]excel middle low Source @MonsterChenzhuo
102 [File]JSON low low Sink @hailin0 Done
103 [File]JSON low low Source @TyrantLucifer Done
104 MaxCompute middle middle Source @longer-jl
105 MaxCompute middle middle Sink @longer-jl
106 TDSql middle middle Source @dzzxjl
107 OpenMLDB middle high Source @TyrantLucifer
107 OpenMLDB middle high Sink @Dlimeng
108 Ftp middle middle Sink @chessplay done #2774
109 Ftp middle middle Source @guanboo done #2774
110 GaussDB middle middle Source @Builder34
111 GaussDB middle middle Sink @Builder34
110 Teradata middle middle Source
111 Teradata middle middle Sink
112 SFTP middle middle Sink @TyrantLucifer
113 SFTP middle middle Source @TyrantLucifer
114 DataHub middle middle Source @selectbook
115 DataHub middle middle Sink @chessplay done
116 SAP HANA middle middle Source
117 SAP HANA middle middle Sink
118 Flink Table Store high high Sink @iture123
119 Flink Table Store high high Source @zhaomin1423
120 Vertica middle middle Source
121 Vertica middle middle Sink
122 Kylin middle middle Source @531651225
123 Kylin middle middle Sink @TaoZex
124 Neocrm middle middle Source
125 TiDB middle middle Source @Xuxiaotuan #2830
126 TiDB middle middle Sink @Xuxiaotuan #2830
127 Sentry middle middle Source
128 PolarDB middle middle Source
129 PolarDB middle middle Sink
130 PolarDB-X middle middle Source
131 PolarDB-X middle middle Sink
132 AnalyticDB middle middle Sink
133 TDSQL middle middle Sink
134 SequoiaDB middle middle Sink
135 TcaplusDB middle middle Source
136 TcaplusDB middle middle Sink
137 GoldenDB middle middle Source
138 GoldenDB middle middle Sink
139 AntDB middle middle Source
140 AntDB middle middle Sink
141 OushuDB middle middle Sink
142 SUNDB middle middle Source
143 SUNDB middle middle Sink
144 UXDB middle middle Source
145 UXDB middle middle Sink
146 DolphinDB middle middle Source
147 DolphinDB middle middle Sink
148 RapidsDB middle middle Source
149 RapidsDB middle middle Sink
150 GreatDB middle middle Source
151 GreatDB middle middle Sink
152 CirroData middle middle Sink
153 Nebula middle middle Sink
154 Gbase 8a middle middle Source
155 KunlunDB middle middle Sink
156 Percona middle middle Source
157 Percona middle middle Sink
158 Splunk middle middle Source
159 Splunk middle middle Sink
160 Amazon DynamoDB middle middle Source
161 Amazon DynamoDB middle middle Sink
162 Microsoft Azure SQL Database middle middle Source
163 Microsoft Azure SQL Database middle middle Sink
164 Neo5j middle middle Source
165 Neo5j middle middle Sink
166 Solr middle middle Sink
167 BigQuery middle middle Source
168 BigQuery middle middle Sink
169 SAP Adaptive Server middle middle Source
170 SAP Adaptive Server middle middle Sink
171 Microsoft Azure Cosmos DB middle middle Source
172 Microsoft Azure Cosmos DB middle middle Sink
173 PostGIS middle middle Source
174 PostGIS middle middle Sink
175 Couchbase middle middle Sink
176 Vika middle middle Sink
177 Gitlab low low Source
EricJoy2048 commented 2 years ago

Great job! I can work for Hive Sink Connector.

ashulin commented 2 years ago

I can work for Apache pulsar source connector.

ic4y commented 2 years ago

Great. Can you assign me to MongoDB connector.

qigengwei commented 2 years ago

Good job!

ic4y commented 2 years ago

Sorry i plan to do JDBC Sink first. Please reassign JDBC Sink to me

wolfboys commented 2 years ago

Hbase, Redis thx.

EricJoy2048 commented 2 years ago

Since most of the work in hive's sink connector is the writing of HDFS files, which is very similar to the work in file sink, I also complete the file sink connector.

zhuangchong commented 2 years ago

socket source, thx.

1999

zhuangchong commented 2 years ago

Http source thx.

ic4y commented 2 years ago

jdbcSink is done, please assign jdbcSource to me. Thanks

legendtkl commented 2 years ago

Socket sink

wuchunfu commented 2 years ago

MongoDB thx

davidzollo commented 2 years ago

good job, has assigned to you

MongoDB thx

hk-lrzy commented 2 years ago

i can work for doris sink first. can you assign it to me?

iture123 commented 2 years ago

I can work for ElasticSearch Source and Sink Connector.

it is too dependent now,because including transport client、RestHighLevelClient and RestLowLevelClient in Seatunnel.We can use RestLowLevelClient to support all versions of ElastichSearch.

2013650523 commented 2 years ago

Kudu thx.

Emor-nj commented 2 years ago

I can work for hudi Source and Sink Connector. thx

EricJoy2048 commented 2 years ago

@Hisoka-X Can we split file sink into HdfsFileSink, LocalFileSink and S3FileSink or other? I implemented HdfsFileSink and LocalFileSink, and I hope some people can implement S3FileSink.

Hisoka-X commented 2 years ago

@Hisoka-X Can we split file sink into HdfsFileSink, LocalFileSink and S3FileSink or other? I implemented HdfsFileSink and LocalFileSink, and I hope some people can implement S3FileSink.

Got it

lhyundeadsoul commented 2 years ago

I can work for tidb Source and Sink Connector. thx

Can you add Assert sink ? @Hisoka-X thx

Hisoka-X commented 2 years ago

I can work for tidb Source and Sink Connector. thx

Can you add Assert sink ? @Hisoka-X thx

Got it

2013650523 commented 2 years ago

I can work for s3 Sink Connector. thx

CalvinKirs commented 2 years ago

I can work for s3 Sink Connector. thx

welcome:) assigned to you.

ovasty commented 2 years ago

Need Apache IoTDB, thanks

FlechazoW commented 2 years ago

I can work for pulsar sink connector. thx

FlechazoW commented 2 years ago

How about add TD-engine?

EricJoy2048 commented 2 years ago

How about add TD-engine?

I think it's a good idea.

EricJoy2048 commented 2 years ago

Need Apache IoTDB, thanks

cc @Hisoka-X

EricJoy2048 commented 2 years ago

I can work for pulsar sink connector. thx

You can welcome. cc @Hisoka-X

lhyundeadsoul commented 2 years ago

@EricJoy2048 @Hisoka-X @CalvinKirs I'm already in touch with TDengine guys. So please give it to me.

2013650523 commented 2 years ago

i can work for Iceberg source and sink connector. can you assign it to me?

Hisoka-X commented 2 years ago

i can work for Iceberg source and sink connector. can you assign it to me?

Great, I assgined to you

2013650523 commented 2 years ago

i can work for doris source connector. can you assign it to me?

Hisoka-X commented 2 years ago

i can work for doris source connector. can you assign it to me?

I will assgine to you.

legendtkl commented 2 years ago

hi, @Hisoka-X , forgot to update that socket seems to not support sink. The task is not needed.

Socket sink

TyrantLucifer commented 2 years ago

Feishu sink and source, thx.

Hisoka-X commented 2 years ago

Feishu sink and source, thx.

Good job!👍

EricJoy2048 commented 2 years ago

i can work for doris sink first. can you assign it to me?

Hi, are there any new updates about this issue？Are there any difficulties that need the help of the community？

TyrantLucifer commented 2 years ago

oss sink and source, thx

EricJoy2048 commented 2 years ago

oss sink and source, thx

cc @Hisoka-X

2013650523 commented 2 years ago

i can work for s3 source . can you assign it to me?

2013650523 commented 2 years ago

i can work for doris sink first. can you assign it to me?

Hi, are there any new updates about this issue？Are there any difficulties that need the help of the community？

Thank you. There is no problem for now. If there is any problem, I will consult with you automatically

Hisoka-X commented 2 years ago

i can work for s3 source . can you assign it to me?

Great

ic4y commented 2 years ago

@Hisoka-X Please add JDBC-DM database. Relate #2377

hailin0 commented 2 years ago

I can work for [File]JSON sink connector. thx

hailin0 commented 2 years ago

I can work for File(HDFS) source connector. thx

TyrantLucifer commented 2 years ago

I can work for File(HDFS) source connector. thx

Hi bro, I’m so sorry, I forgot to comment on the issue, this connector I have developed more than halfway. What's your progress so far?

hailin0 commented 2 years ago

I can work for File(HDFS) source connector. thx

Hi bro, I’m so sorry, I forgot to comment on the issue, this connector I have developed more than halfway. What's your progress so far?

Ok, I haven't started coding

hailin0 commented 2 years ago

I can work for IoTDB source/sink connector. thx

CalvinKirs commented 2 years ago

I can work for IoTDB source/sink connector. thx

hi, I forgot to remark, I am starting to do Source here, how about we do it(I do source and you do sink) together?