search

redpanda-data / connect

Fancy stream processing made operationally mundane
https://docs.redpanda.com/redpanda-connect/about/
8.14k stars 840 forks source link

Couchbase as source and sink #364

Open aadicodes opened 4 years ago

aadicodes commented 4 years ago

Are there any plans to add Couchbase as Source and/or Sink ?

aadicodes commented 4 years ago

I started thinking whether Couchbase suits as Source. Probably its a good fit for being just a Sink. any thoughts ?

Jeffail commented 4 years ago

Hey @aadicodes, I think Couchbase support makes sense. There's an official client lib that looks well supported https://github.com/couchbase/gocb.

In terms of how we'd integrate it I think an output similar to the existing Elasticsearch one makes sense, and it would be cool as well to add it as a cache type at some point as well.

sapk commented 1 year ago

I think I can implement cache, input, output and processor similar to what the mongodb implementation does. But I think maybe step by step and start with processor. Do you have example of what you have expect as an input and output ?

mihaitodor commented 1 year ago

I think those should end up looking similar to the CosmosDB ones that I'm adding in #1583.

Mizaro commented 6 months ago

Is this issue considered taken? 😇

mihaitodor commented 6 months ago

@Mizaro I don't think anyone is working on it. PRs are welcome.

pederbl commented 2 months ago

We would love to migrate from Kafka Connect to Redpanda Connect but we are dependent on having source and sink support. Our use case is to make it simpler for our hackathon participants to build integrations and graphql subscriptions. Our platform, cillers.com, uses Couchbase as operational data store.

sapk commented 2 months ago

I don't know how you plan to use it but the processor can be used as an output already with: https://docs.redpanda.com/redpanda-connect/components/processors/about/#using-processors-as-outputs.

Otherwise, the example given here for input or output would be more based on n1sql which is not supported in the current logic and it should probably be integrated as a standard sql driver with generic sql inputs and outputs instead of a dedicated input or output.

The current logic only support the kv part of couchbase as a processor or a cache.

larsenpanda commented 3 weeks ago

This is done!