search

openfaas / of-watchdog

Reverse proxy for STDIO and HTTP microservices
MIT License
259 stars 115 forks source link

Support issue for Python template #110

Closed afilipov1 closed 3 years ago

afilipov1 commented 3 years ago

Expected Behaviour

A way to configure how many threads of the underline WSGI server to be used for function execuiton

Current Behaviour

Default threads count is 4 and thus only 4 parallel functions are executed at the same time

Possible Solution

Providing an env variable inside templates

USER app

ENV fprocess="python index.py"
ENV cgi_headers="true"
ENV mode="http"
ENV upstream_url="http://127.0.0.1:5000"

Steps to Reproduce

Function

version: 1.0
provider:
  name: openfaas
  gateway: http://192.168.64.14:31112
functions:
  hello-openfaas-of-watchdog:
    lang: python3-http
    handler: ./hello-openfaas-of-watchdog
    image: afilipov/hello-openfaas-of-watchdog:latest
    labels:
      com.openfaas.scale.min: 1
      com.openfaas.scale.max: 1
    environment:
      write_debug: true
      read_timeout: 60s
      write_timeout: 60s
      exec_timeout: 180s

import time

def handle(event, context):
    # some_str = ' ' * 1024*300

    print("Start : %s" % time.ctime())
    time.sleep(15)
    print("End : %s" % time.ctime())

    return "Response from of-watchdog"

Testing results with 50 parallel invokations

hey -n=50 -c 50 -m POST -d=Test http://192.168.64.14:31112/function/hello-openfaas-of-watchdog

2020/09/03 10:05:45 OperationalMode: http
2020/09/03 10:05:45 Timeouts: read: 1m0s, write: 1m0s hard: 3m0s.
2020/09/03 10:05:45 Listening on port: 8080
2020/09/03 10:05:45 Writing lock-file to: /tmp/.lock
2020/09/03 10:05:45 Metrics listening on port: 8081
2020/09/03 10:06:24 POST / - 200 OK - ContentLength: 0
2020/09/03 10:08:00 stderr: WARNING:waitress.queue:Task queue depth is 1
2020/09/03 10:08:00 stderr: WARNING:waitress.queue:Task queue depth is 2
2020/09/03 10:08:00 stderr: WARNING:waitress.queue:Task queue depth is 3
2020/09/03 10:08:00 stderr: WARNING:waitress.queue:Task queue depth is 4
2020/09/03 10:08:00 stderr: WARNING:waitress.queue:Task queue depth is 5
2020/09/03 10:08:00 stderr: WARNING:waitress.queue:Task queue depth is 6
2020/09/03 10:08:00 stderr: WARNING:waitress.queue:Task queue depth is 7
2020/09/03 10:08:00 stderr: WARNING:waitress.queue:Task queue depth is 8
2020/09/03 10:08:00 stderr: WARNING:waitress.queue:Task queue depth is 9
2020/09/03 10:08:00 stderr: WARNING:waitress.queue:Task queue depth is 10
2020/09/03 10:08:00 stderr: WARNING:waitress.queue:Task queue depth is 11
2020/09/03 10:08:00 stderr: WARNING:waitress.queue:Task queue depth is 12
2020/09/03 10:08:00 stderr: WARNING:waitress.queue:Task queue depth is 13
2020/09/03 10:08:00 stderr: WARNING:waitress.queue:Task queue depth is 14
2020/09/03 10:08:00 stderr: WARNING:waitress.queue:Task queue depth is 15
2020/09/03 10:08:00 stderr: WARNING:waitress.queue:Task queue depth is 16
2020/09/03 10:08:00 stderr: WARNING:waitress.queue:Task queue depth is 17
2020/09/03 10:08:00 stderr: WARNING:waitress.queue:Task queue depth is 18
2020/09/03 10:08:00 stderr: WARNING:waitress.queue:Task queue depth is 19
2020/09/03 10:08:00 stderr: WARNING:waitress.queue:Task queue depth is 20
2020/09/03 10:08:00 stderr: WARNING:waitress.queue:Task queue depth is 21
2020/09/03 10:08:00 stderr: WARNING:waitress.queue:Task queue depth is 22
2020/09/03 10:08:00 stderr: WARNING:waitress.queue:Task queue depth is 23
2020/09/03 10:08:00 stderr: WARNING:waitress.queue:Task queue depth is 24
2020/09/03 10:08:00 stderr: WARNING:waitress.queue:Task queue depth is 25
2020/09/03 10:08:00 stderr: WARNING:waitress.queue:Task queue depth is 26
2020/09/03 10:08:00 stderr: WARNING:waitress.queue:Task queue depth is 27
2020/09/03 10:08:00 stderr: WARNING:waitress.queue:Task queue depth is 28
2020/09/03 10:08:00 stderr: WARNING:waitress.queue:Task queue depth is 29
2020/09/03 10:08:00 stderr: WARNING:waitress.queue:Task queue depth is 30
2020/09/03 10:08:00 stderr: WARNING:waitress.queue:Task queue depth is 31
2020/09/03 10:08:00 stderr: WARNING:waitress.queue:Task queue depth is 32
2020/09/03 10:08:00 stderr: WARNING:waitress.queue:Task queue depth is 33
2020/09/03 10:08:01 stderr: WARNING:waitress.queue:Task queue depth is 34
2020/09/03 10:08:01 stderr: WARNING:waitress.queue:Task queue depth is 35
2020/09/03 10:08:01 stderr: WARNING:waitress.queue:Task queue depth is 36
2020/09/03 10:08:01 stderr: WARNING:waitress.queue:Task queue depth is 37
2020/09/03 10:08:01 stderr: WARNING:waitress.queue:Task queue depth is 38
2020/09/03 10:08:02 stderr: WARNING:waitress.queue:Task queue depth is 39
2020/09/03 10:08:02 stderr: WARNING:waitress.queue:Task queue depth is 40
2020/09/03 10:08:02 stderr: WARNING:waitress.queue:Task queue depth is 41
2020/09/03 10:08:02 stderr: WARNING:waitress.queue:Task queue depth is 42
2020/09/03 10:08:02 stderr: WARNING:waitress.queue:Task queue depth is 43
2020/09/03 10:08:03 stderr: WARNING:waitress.queue:Task queue depth is 44
2020/09/03 10:08:03 stderr: WARNING:waitress.queue:Task queue depth is 45
2020/09/03 10:08:03 stderr: WARNING:waitress.queue:Task queue depth is 46
2020/09/03 10:08:15 POST / - 200 OK - ContentLength: 0
2020/09/03 10:08:15 POST / - 200 OK - ContentLength: 0
2020/09/03 10:08:15 POST / - 200 OK - ContentLength: 0
2020/09/03 10:08:15 POST / - 200 OK - ContentLength: 0
2020/09/03 10:08:30 POST / - 200 OK - ContentLength: 0
2020/09/03 10:08:30 POST / - 200 OK - ContentLength: 0
2020/09/03 10:08:30 POST / - 200 OK - ContentLength: 0
2020/09/03 10:08:30 POST / - 200 OK - ContentLength: 0

From the last 2 blocks of logs we can see that only 4 requests are processed at the same time. Tested the same scenario with the classic watchdog and the execution time is much lower because of the multiple parallel processes.

  PID  PPID USER     STAT   VSZ %VSZ CPU %CPU COMMAND
    1     0 app      S     108m   6%   1   1% fwatchdog
 1716     1 app      S     8232   0%   0   0% python3 index.py
 1718     1 app      S     8232   0%   1   0% python3 index.py
 1717     1 app      S     8232   0%   1   0% python3 index.py
 1722     1 app      S     8232   0%   0   0% python3 index.py
 1719     1 app      S     8232   0%   0   0% python3 index.py
 1720     1 app      S     8232   0%   0   0% python3 index.py
 1721     1 app      S     8232   0%   0   0% python3 index.py
 1806     1 app      S     8232   0%   1   0% python3 index.py
 1723     1 app      S     8232   0%   1   0% python3 index.py
 1724     1 app      S     8232   0%   1   0% python3 index.py
 1725     1 app      S     8232   0%   1   0% python3 index.py
 1731     1 app      S     8232   0%   0   0% python3 index.py
 1810     1 app      S     8232   0%   1   0% python3 index.py
 1789     1 app      S     8232   0%   1   0% python3 index.py
 1788     1 app      S     8232   0%   0   0% python3 index.py
 1735     1 app      S     8232   0%   1   0% python3 index.py
 1734     1 app      S     8232   0%   0   0% python3 index.py
 1742     1 app      S     8232   0%   0   0% python3 index.py
 1736     1 app      S     8232   0%   1   0% python3 index.py
 1744     1 app      S     8232   0%   0   0% python3 index.py
 1745     1 app      S     8232   0%   1   0% python3 index.py
 1746     1 app      S     8232   0%   0   0% python3 index.py
 1743     1 app      S     8232   0%   1   0% python3 index.py
 1755     1 app      S     8232   0%   1   0% python3 index.py
 1726     1 app      S     8232   0%   1   0% python3 index.py
 1733     1 app      S     8232   0%   0   0% python3 index.py
 1758     1 app      S     8232   0%   0   0% python3 index.py
 1771     1 app      S     8232   0%   0   0% python3 index.py

Context

Currently we are evaluating the benefits for us if migrating to the new of-watchdog. We are looking for high concurrency and throughput with the new watchdog. With simple function without sleep the tests shows significant improvements with the new of-watchdog.

Your Environment

alexellis commented 3 years ago

Hi @AntonioFilipov

I'm not sure this is anything to do with the watchdog, it sounds like an issue with the python3-http template that you're using and the behaviour of its HTTP server called waitress.

I'd suggest you spend a bit of time with their documentation and browsing the template repo below:

You can find the repo here: https://github.com/openfaas-incubator/python-flask-template

For now I'll close this issue since it's attributing the behaviour to the of-watchdog which I believe to be incorrect (happy to be proven wrong of course).

Alex

alexellis commented 3 years ago

/set title: Support issue for Python template

alexellis commented 3 years ago

It seems like you may want the setting in the Waitress docs:

--threads=INT

https://docs.pylonsproject.org/projects/waitress/en/stable/runner.html#runner

If it can configured via ENV-var, then that'd be your easiest option, otherwise feel free to experiment by forking the templates repo, changing them and testing them out.

faas-cli template pull https://github.com/your-repo/templates

You should also configure some minimum level of availability for the function, like a min scale value, and then consider using the endpoint loadbalancing feature in openfaas, to make sure the replicas are getting an even balance and to prevent any keep-alive code in your testing app.

Endpoint load-balancing in the docs: https://docs.openfaas.com/architecture/production/#endpoint-load-balancing