search

argoproj / argo-events

Event-driven Automation Framework for Kubernetes
https://argoproj.github.io/argo-events/
Apache License 2.0
2.37k stars 738 forks source link

Sensor high memory consuming / Low event processing throughput #3156

Closed artem-fedorov closed 2 months ago

artem-fedorov commented 5 months ago

Argo events use case: Event source - webhook Sensor - k8s actions (create/delete resources, e.g. pod/secrets/job) There are less than 10 triggers in configuration (https://github.com/argoproj/argo-events/issues/1339)

Default Sensor memory utilization ~60-100mb. Side effect: when there are a lot of k8s Secrets in the cluster (e.g.8-20k) Sensor pod starts to consume more memory *up to 500mb. Another issue, in this case, is low event processing throughput (up to 5min between sending webhook until k8s action will be performed). Is there a chance that Sensor persists the list of secrets in memory and does not destroy it? Or how memory consumption may affect throughput?

Environment (please complete the following information):

Message from the maintainers:

If you wish to see this enhancement implemented please add a 👍 reaction to this issue! We often sort issues this way to know what to prioritize.

github-actions[bot] commented 3 months ago

This issue has been automatically marked as stale because it has not had any activity in the last 60 days. It will be closed if no further activity occurs. Thank you for your contributions.