redpanda-data / connect

Fancy stream processing made operationally mundane
https://docs.redpanda.com/redpanda-connect/about/
8.14k stars 840 forks source link

Refusing redis connection #3024

Closed amrubal01 closed 1 day ago

amrubal01 commented 1 day ago

I am using Redpanda Connect (formerly known as Benthos) for data streaming and manipulation. I am trying to output the results of my data streaming to a redis stream. I was previously able to do this, but since the switch to Redpanda I have not been able to connect to Redis even though I have a redis server running locally.

Here is the output mapping I have set up for it: output: redis_streams: url: tcp://redis:6379 stream: "newstream" body_key: body max_length: 0 max_in_flight: 64 metadata: exclude_prefixes: [] batching: count: 0 byte_size: 0 period: "" check: ""

Here is the error I get: configuration file read error: parseLogs.yaml: (33,1) unable to infer output type from candidates: [redis_streams]

I am not sure if there is something I am missing now but I have not been able to find anything in the docs thus far.

https://docs.redpanda.com/redpanda-connect/components/outputs/redis_streams/?tab=tabs-1-common

mihaitodor commented 1 day ago

Hey @amrubal01 👋 I just tried running the following yaml and it works for me:

input:
  stdin: {}
output:
  redis_streams:
    url: tcp://redis:6379
    stream: "newstream"
    body_key: body
    max_length: 0
    max_in_flight: 64
    metadata:
      exclude_prefixes: []
    batching:
      count: 0
      byte_size: 0
      period: ""
      check: ""

Can you please make sure that you're using a Connect release? Alternatively, if you have custom plugins, you'll need to import the appropriate Connect bundle: https://github.com/redpanda-data/redpanda-connect-plugin-example/blob/d1461319dd7f4179bf9095c2cc7b25e8e87239e3/main.go#L8-L13

PS: Moving to a discussion as per #2026.