Is your feature request related to a problem? Please describe.
CDC currently only support emitting the entire message in a specific format using WITH format=.... But some customers would like to encode the key and values in different formats since this is how their downstream apps use the keys and values.
Describe the solution you'd like
Naively thinking about this, we could allow this customization and pass a different encoder or decoder based on the configurations.
blathers-crl[bot]
commented
5 months ago
Is your feature request related to a problem? Please describe. CDC currently only support emitting the entire message in a specific format using
WITH format=.... But some customers would like to encode the key and values in different formats since this is how their downstream apps use the keys and values.Describe the solution you'd like Naively thinking about this, we could allow this customization and pass a different encoder or decoder based on the configurations.
https://github.com/cockroachdb/cockroach/blob/1b4f26ed1210e0d5924e9626629737a595c54118/pkg/ccl/changefeedccl/encoder.go#L23-L37 and here https://github.com/cockroachdb/cockroach/blob/c7264978e40967bb73d24667c2574c23a83ea47c/pkg/ccl/changefeedccl/event_processing.go#L428-L435.
Additional context This feature is a customer request (see more details in https://github.com/cockroachlabs/support/issues/2794).
Jira issue: CRDB-35637