search

Trendyol / go-dcp-kafka

The Go implementation of the Couchbase to Kafka with DCP.
MIT License
67 stars 15 forks source link

When dcp group name includes . (dot), couchbase saves it as subdocument #67

Closed kaygoren closed 10 months ago

kaygoren commented 1 year ago

We gave groupName as globalplatforms.wholesale-cb-kafka-connector, and it is occured as a subdocument in the couchbase checkpoint collection.

image

We need to develop to handle escape characters

oguzyildirim commented 1 year ago

Hello @kaygoren which go-dcp-kafka version are you using?

kaygoren commented 1 year ago

@oguzyildirim hi, we are using v1.1.0 which is currently latest, sorry for the missing info.

oguzyildirim commented 12 months ago

Greetings @kaygoren. In our checkpoint management process, we're making use of Couchbase's sub-document feature. This functionality allows us to work with sub-documents using a path syntax. As you pointed out, when a "." is included in the group name, Couchbase generates a sub-document automatically. To achieve your desired outcome, we'll need to make some modifications during generation of key, such as replacing dots with commas or semicolons. What do you guys think about it @mhmtszr @Abdulsametileri @erayarslan?

erayarslan commented 10 months ago

thanks to @oguzyildirim, we fixed with https://github.com/Trendyol/go-dcp-kafka/commit/42e8ef4158703fce79c5ae07869e4747768252d2