argoproj / argo-events

Event-driven Automation Framework for Kubernetes
https://argoproj.github.io/argo-events/
Apache License 2.0
2.37k stars 738 forks source link

Jetstream Support for NATS EventSource and Sensor #2409

Closed joelcomp1 closed 1 year ago

joelcomp1 commented 1 year ago

Is your feature request related to a problem? Please describe. I can't see any way to use the existing NATS Event Source or Sensor to connect to a jetstream subject.

Describe the solution you'd like A way to update the existing EventSource / Sensor with a jetstream flag, maybe as simple as defining the stream. If left undefined then would fall back to NATS

Describe alternatives you've considered Not using Argo Events since I can't use it to trigger on Jetstream messages.

Additional context I can't see if this is already implemented anywhere but it doesn't seem to be supported.


Message from the maintainers:

If you wish to see this enhancement implemented please add a 👍 reaction to this issue! We often sort issues this way to know what to prioritize.

justinfx commented 5 months ago

Hey @joelcomp1, why was this request completed back in Jan 2023? I didn't see any introduction of this feature and I am now about to create my own new request for it. Was there some external discussion where the maintainers said they didn't want to move forward with it? Or did you decide to just not use Argo Events because of the lack of the EventSource?

joelcomp1 commented 5 months ago

The feature already exists in argo events its just not well documented. i.e.

apiVersion: argoproj.io/v1alpha1
kind: Sensor
metadata:
  name:  test
spec:
  template:
    container:
      imagePullPolicy: IfNotPresent
  dependencies:
    - name: test
      eventSourceName: webhooks
      eventName: test
  triggers:
    - template:
        name: test
        nats:
          url: nats://nats-nats-cluster.nats:4222
          subject: test
          stream: blah
          payload:
            - src:
                dependencyName: test
                dataKey: body
              dest: body

hopefully that helps!

justinfx commented 5 months ago

@joelcomp1 thanks for the reply, but this actually doesn't help. Your original issues asked about both EventSource and Sensor support. I see you have given an example of supposed Sensor support only, which means you would want to take a source event and publish it to nats. Not only is the "stream" argument not documented. I don't even see it in the nats trigger source code: https://github.com/argoproj/argo-events/blob/master/sensors%2Ftriggers%2Fnats%2Fnats.go

So the stream param just seems to be ignored? And even if the "stream" parameter were supported, you don't really need to specify the stream name when publishing, unless you want some kind of validation from nats that the subject matches the stream. It should get stored in the stream either way.

So that leaves the question as to whether you actually wanted EventSource support? Did you want to consume from nats streams?