datastrato / gravitino

World's most powerful data catalog service with providing a high-performance, geo-distributed and federated metadata lake.
https://datastrato.ai/docs/
Apache License 2.0
401 stars 166 forks source link

[#3362] feat(flink-connector): Add the code skeleton for flink-connector #2635

Closed coolderli closed 1 week ago

coolderli commented 2 months ago

What changes were proposed in this pull request?

Why are the changes needed?

Does this PR introduce any user-facing change?

How was this patch tested?

jerryshao commented 2 weeks ago

@coolderli can we please move forward the flink support?

coolderli commented 2 weeks ago

@coolderli can we please move forward the flink support?

@jerryshao Of course. I will finish the first one today.

coolderli commented 2 weeks ago

@jerryshao @FANNG1 Could you please help view this? Thanks.

coolderli commented 2 weeks ago

@FANNG1 I have updated the MR. Please help review it when you have time. Thanks.

coolderli commented 1 week ago

@FANNG1 Could you help review this MR again? Thanks.

coolderli commented 1 week ago

@FANNG1 I think I have already rebased the main branch. But the GitHub CI still failed. Can you help me with it? Thanks.

FANNG1 commented 1 week ago

@FANNG1 I think I have already rebased the main branch. But the GitHub CI still failed. Can you help me with it? Thanks.

seems CI failure are cause by #3390 which change the API about metalake and catalog operations. You could refer the latest the implement of Spark GravitinoCatalogManager for the usage.

coolderli commented 1 week ago

@FANNG1 I have fixed the pipeline. Please help review again. Thanks.

coolderli commented 1 week ago

@FANNG1 Address all comments. Please review it again. Thanks a lot.

FANNG1 commented 1 week ago

LGTM, except few comments, @jerryshao do you have time to review the PR?

jerryshao commented 1 week ago

@FANNG1 You can go ahead if you feel OK. Thanks @coolderli for the first step of Flink support.

FANNG1 commented 1 week ago

@coolderli , thanks for proposing the base PR for Flink, a big step!