search

getindata / flink-http-connector

Http Connector for Apache Flink. Provides sources and sinks for Datastream , Table and SQL APIs.
Apache License 2.0
150 stars 43 forks source link

What's the meaning of ESP-xx in git message? #31

Closed ChengkaiYang2022 closed 1 year ago

ChengkaiYang2022 commented 1 year ago

Hi, I like your work here!

I find that you list some interesting TODOs in the README.md page , and I want to contribute some help to this repository. Actually, I tried to do my own version of flink-http-connector last months ago, but I only finished table sink part. I guess it would be better for me to contribute in this project, instead of working on my own project, since you already did so many amazing works!

Anyway, I note that you used ESP-98-xxx in the git message, is ESP-98-xxx like a JIRA issue of this project? (Like Apache Flink use FLINK-19999 in their git message to link the JIRA issue)? Would you mind to share the details about these ESP ? Or is there a JIRA page for this project that I can learn more about the history of this project?

ChengkaiYang2022 commented 1 year ago

Ok, I noticed that there are some descriptions and discussion under the PR page.

Anyway, just wonder, what ESP stand for? :)

kristoffSC commented 1 year ago

Hi @ChengkaiYang2022 I'm very happy to see you being interested with our little project here :)

I will give you more info on Monday, now just want to say "welcome" :)

kristoffSC commented 1 year ago

Hi @ChengkaiYang2022

I'm very happy to see you being interested with our little project here :)

I will try to answer your questions. Http-connector started from HTTP Lookup Source. The main requirement for this was to allow users to express "enrichment" from HTTP endpoints using SQL JOIN and SQL tables, without UDF. After that (when 1.15 Flink was released) we started implementing HTTP Sink that is based on Flink's 1.15 Async Sink.

As for Jira question - we don't have any dedicated JIRA for this connector. I think for now we should "Github issues" to drive development at least for now.

Originally main functional requirements for this connector were coming from one of our commercial projects. This project is not public and it is tracked using internal JIRA. The 'ESP' is code name from this internal JIRA. Since currently all active developers are/were from our company we all had access there. Unfortunately there is no way to open access to this instance.

What we should have done is that we should mimic those Jira tickets as Github Issues for connector repository - lessons learned. We will start doing this from now. In that case the commit message could be prefixed with [HTTP-xxx] where XXX would be number of Hithub issue. The good news is that currently it seems we are all set with functionalities and what we have as TODO's are enhancements not strictly related with that commercial project.

The project is fresh/young so we don't have all things written down as it comes for development process although we have few things "established" already. I think I will try to find some time and wrote those down.

As it comes for contribution. As you saw we have couple of TODO's that we think would be good to implement. I think the first step would be to create github issue as enhancement/discussion and start discussion the implementation ideas. After we finalize the design we can create implementation/enhancement issue with proper description what should be implemented and assign it to the developer.

ChengkaiYang2022 commented 1 year ago

@kristoffSC Thanks for the reply!. I get the meaning of 'ESP' after the reading the comment.

I agree with the idea that using "Github issues" to drive development and using '[HTTP-xxx]' in the git message to address the github issue.

And I like the idea of beginning with creating github issue and discussion, just like the Apache brief "Community over code".

Okay, I will check more details of this project and try to get a better understanding of "Flink's 1.15 Async Sink." firstly.

Anyway, It's so nice to find this project!