search

couchbase / kafka-connect-couchbase

Kafka Connect connector for Couchbase Server
https://issues.couchbase.com/projects/KAFKAC
Apache License 2.0
73 stars 72 forks source link

Regarding KIP-875 `alterOffsets` #45

Open yeikel opened 1 month ago

yeikel commented 1 month ago

Hi team,

According to KIP-875, when a connector does not implement alterOffsets, the result of resetting the offsets is expected to be

{
  "message": "The framework-managed offsets for this connector have been <reset|altered> successfully. However, if this connector manages offsets externally, they will need to be manually <altered|reset> in the system that the connector uses."
}

This is exactly what I am seeing while trying to reset the offsets of the source connector.

If the connector is not expected to support this, a clarification in the docs will also be appreciated

dnault commented 1 month ago

The Couchbase connector does not manage offsets externally, so there should be no problem (apart from the scary warning message).

@yeikel Do you have an opinion on where in the documentation you'd expect to see this mentioned?

To make the warning go away, we would need to compile against Kafka 3.6.0. I think we might be able to do that in a way that retains compatibility with the current minimum version (2.8), but we'd need to test carefully.

yeikel commented 3 weeks ago

@yeikel Do you have an opinion on where in the documentation you'd expect to see this mentioned?

I don't. It seems that the docs do not have any clear section abut this right now and maybe we need to add a warning/new section about it

To make the warning go away, we would need to compile against Kafka 3.6.0. I think we might be able to do that in a way that retains compatibility with the current minimum version (2.8), but we'd need to test carefully.

That makes sense. If that's the case then it is best not to for now as I suspect that compatibility with older versions is important