numaproj / numaflow

Kubernetes-native platform to run massively parallel data/streaming jobs
https://numaflow.numaproj.io/
Apache License 2.0
1.13k stars 113 forks source link

Map Udf stops consuming without any errors #1542

Open akashkhamkarr opened 8 months ago

akashkhamkarr commented 8 months ago

Describe the bug We have noticed that in certain pipelines, the map udf stops consuming the events from the ISB completely. This behaviour is intermittent, and we couldn’t find any error logs on numa as well as the udf container. Usually after restarting, the pipelines resumes working.

To Reproduce Not reproducible

Expected behavior Map udf shouldn’t stop consuming, and stall the pipeline without any errors. At least we need more visibility on what might be wrong.

Screenshots

Screenshot 2024-03-05 at 9 41 38 AM

Numa container logs for this udf numa-udf-stall.txt

Environment (please complete the following information):

Additional context

For quick help and support, join our slack channel.

Message from the maintainers:

Impacted by this bug? Give it a 👍. We often sort issues this way to know what to prioritize.

vigith commented 7 months ago

is this still an issue?