googleapis / google-cloud-go

Google Cloud Client Libraries for Go.
https://cloud.google.com/go/docs/reference
Apache License 2.0
3.8k stars 1.3k forks source link

spanner: change streams reading support #10790

Open jferrl opened 2 months ago

jferrl commented 2 months ago

Is your feature request related to a problem? Please describe.

It would be easier to work with Spanner’s change streams if the google-cloud-go Spanner client provided built-in support for easy reading of these streams. Currently, developers must manually implement logic to handle change streams, which adds complexity and potential for errors. Having native support within the client library would streamline development and reduce the likelihood of issues.

Describe the solution you'd like

I would like the google-cloud-go Spanner client to include an API that simplifies the process of reading change streams

Describe alternatives you've considered

One alternative is to build custom solutions using the existing Spanner client to read and process change streams. However, this approach requires more boilerplate code and in-depth knowledge of how change streams work, which can lead to increased development time and maintenance complexity. Other option could be the usage of https://github.com/cloudspannerecosystem/spanner-change-streams-tail/tree/main/changestreams pkg

Additional context

Integrating change stream support directly into the google-cloud-go Spanner client would reduce the effort needed to work with Spanner in real-time data scenarios.

alarbada commented 2 months ago

I support this, I'm currently building a conduit connector for reading spanner change streams and it is quite painful.