apache / pekko-connectors

Apache Pekko Connectors is a Reactive Enterprise Integration library for Java and Scala, based on Reactive Streams and Apache Pekko.
https://pekko.apache.org/
Apache License 2.0
56 stars 27 forks source link

Add support for couchbase3 client #623

Open laglangyue opened 2 months ago

laglangyue commented 2 months ago

I spent about 6 weekends (30 hours) trying to transitie Couchbase2 to Couchbase3 smoothly. In the end, I basically completed the code migration, but it became very ugly and difficult to use and test, so I closed that PR.

400

503

504

I spent about 3 days rewriting the connector. Mainly referring to the mongo connector. The new connector is similar to the Mongo connector, with only API packaging for the SDK. The couchbase-SDK is already very powerful. Couchbase-sdk also provides APIs for mono and flux (based on the built-in Completufurture). I also design the API of pekko-connector-3 with the async api(Completufuture)

I just did two tests, and I will complete the rest and docs next week or Labor Day.

Source api

Flow api

Sink api

laglangyue commented 2 months ago

Community members can take a look at the PR first. Although it is a draft, the API part has been completed. Next, the main tasks are to complete the annotations, documentation, testing, and testing framework.

Most APIs are based on collection. In Couchbase3, cluster (Connection) ->bucket ->scope ->collection

So I made the collection an implicit

The old couchbase has a SessionKey (setting, bucket) connection pool, which we no longer need. I haven't implemented it yet. The connection is not based on buckets, but on clusters.

I will temporarily put aside the implementation of pekko-couchbase-discovery.

laglangyue commented 2 months ago

@pjfanning @mdedetrich @He-Pin Although the PR has not been completed yet, I hope to receive some of your suggestions. I will improve the implementation based on your suggestions.

He-Pin commented 2 months ago

Thanks for putting a lot into this, I will check this after work.

laglangyue commented 2 months ago

I have basically completed the work on adding couchbase3, but not doc. We can keep the code for couchbase2, but drop the documentation for couchbase2. Write documents based on CouchBase3, adn we declare that we no longer support Couchbase2 in docs, but will continue to release several versions. I separated their modules and it will be easy to remove couchbase2 in the future.

pjfanning commented 1 month ago

I've merged the implementation PR but left this open because we'll need at least some doc changes to reflect the new cloudbase3 connector.

laglangyue commented 1 month ago

I've merged the implementation PR but left this open because we'll need at least some doc changes to reflect the new cloudbase3 connector.

Yeah, I just finished my vacation, but I'm about to go on a business trip and will be doing this PR on this Sunday