Raising crash error in map fn will result in message drop

[sandangel]

Describe the bug A clear and concise description of what the bug is.

To Reproduce Steps to reproduce the behavior:

1. Use Map FN
2. Raise AssertionError in python inside map fn
3. Check the logs and see grpc server dropping that message

Expected behavior A clear and concise description of what you expected to happen.

Message should not be dropped.

Screenshots If applicable, add screenshots to help explain your problem.

More in this slack thread: https://numaproj.slack.com/archives/C03GQ3AMHE1/p1679533681725289

Environment (please complete the following information):

• Kubernetes: 1.23
• Numaflow: latest image, python sdk with latest commit from main

Additional context Add any other context about the problem here.

---

Message from the maintainers:

Impacted by this bug? Give it a 👍. We often sort issues this way to know what to prioritize.

[whynowy]

https://github.com/numaproj/numaflow-python/blob/f6c914e056b2bb4d3f1bd97888ab44b5fe2055c2/pynumaflow/function/server.py#L144-L145

[vigith]

maybe we should catch runtime exception and re-raise it?

[KeranYang]

This also applies to MapT. I am not 100% sure about reduce but we can check when we start working on this issue.

[whynowy]

Reduce as well.