Support request for long running functions

[xiaowei6688]

Hello everyone, I have a problem at present. My function is a tool with time-consuming operations. Each processing takes about 5 or 10 minutes. The following error will occur after I make two consecutive requests at the same time, and the container process will exit. This error occurs whether synchronous or asynchronous.（There is no problem in executing a single request） I don't know why. Please give me some advice. Thank you very much.

2022/09/16 07:14:56 Upstream HTTP request error: Post "http://127.0.0.1:5000/": EOF 2022/09/16 07:14:56 Upstream HTTP request error: Post "http://127.0.0.1:5000/": EOF 2022/09/16 07:14:56 Forked function has terminated: exit status 114

Expected Behaviour

Each function can be successfully executed at the same time (although I don't know whether it can be)

Current Behaviour

2022/09/16 07:14:56 Upstream HTTP request error: Post "http://127.0.0.1:5000/": EOF 2022/09/16 07:14:56 Upstream HTTP request error: Post "http://127.0.0.1:5000/": EOF 2022/09/16 07:14:56 Forked function has terminated: exit status 114

Are you a GitHub Sponsor (Yes/No?)

Check at: https://github.com/sponsors/openfaas

• [ ] Yes
• [√ ] No

Which Solution Do You Recommend?

Please point out what I need to pay attention to or tell me what I need to configure

Context

Each request for a function is ultimately a time-consuming operation. I need two requests to be successfully executed at the same time without the above errors

2022/09/16 07:14:56 stderr: ASSERTION VIOLATION 2022/09/16 07:14:56 stderr: File: ../src/ast/rewriter/rewriter_def.h 2022/09/16 07:14:56 stderr: Line: 226 2022/09/16 07:14:56 stderr: UNEXPECTED CODE WAS REACHED. 2022/09/16 07:14:56 stderr: Z3 4.11.2.0 2022/09/16 07:14:56 stderr: Please file an issue with this message and more detail about how you encountered it at https://github.com/Z3Prover/z3/issues/new 2022/09/16 07:14:56 Upstream HTTP request error: Post "http://127.0.0.1:5000/": EOF 2022/09/16 07:14:56 Upstream HTTP request error: Post "http://127.0.0.1:5000/": EOF 2022/09/16 07:14:56 Forked function has terminated: exit status 114 2022/09/16 07:14:56 stdout: }

Your Environment

faas-cli version 0.14.2 docker version 20.10.14 kubectl version v1.23.1 ubuntu 20.04 template python3-flask-debian

[alexellis]

@welteki please can you ask for any additional information for a repro etc?

[alexellis]

@xiaowei6688 thanks for your interest in OpenFaaS

Unfortunately, we cannot debug your code or OpenFaaS configuration, because this is a form of Enterprise Support.

If you suspect you've found a bug, you'll need to provide a full code sample using something that you don't mind sharing like a sleep function or something else that's long running.

The Python template is multi-threaded, and can support multiple concurrent requests. It seems like an issue with your code (from the logs).

If your container can only handle 1-2 requests, then perhaps set a hard limit for it with the max_inflight setting.

https://github.com/openfaas/of-watchdog

The OpenFaaS Pro queue-worker supports retrying busy functions.

https://docs.openfaas.com/openfaas-pro/retries

Alex

[alexellis]

/set title: Support request for long running functions

[alexellis]

/add label: support, insufficient-info