numaproj / numaflow

Kubernetes-native platform to run massively parallel data/streaming jobs
https://numaflow.numaproj.io/
Apache License 2.0
1.1k stars 112 forks source link

documentation around how the pipeline handles errors (retryable and non-retryable errors) #139

Open shashank10456 opened 2 years ago

shashank10456 commented 2 years ago

Summary

What change needs making?

Documentation around how the pipeline handles errors (retryable and non-retryable errors).

Use Cases

When would you use this?

This will help in understanding the platform better and help the developers in handling the retryable errors in a better way without panicking.


Message from the maintainers:

If you wish to see this enhancement implemented please add a 👍 reaction to this issue! We often sort issues this way to know what to prioritize.

vigith commented 2 years ago

With the new gRPC-based UDF, a user-defined error cannot be returned to the SDK. The numaflow framework will retry any internal error, and the UDF author should retry the errors within the UDF.