function custom readiness checking not working as documented

rgschmitz1 commented 1 year ago

Expected Behaviour

Custom readiness check should allow for routing traffic away from overloaded pods as described here, https://www.openfaas.com/blog/health-and-readiness-for-functions/

Current Behaviour

Attempting to test concurrent execution of a function with multiple replicas deployed results in overloading due to inconsistent load balancing.

Possible Solution

I might try to downgrade the of-watchdog version, currently using v0.9.10 (obviously not a fix but if it resolves my issue i can report back)

Steps to Reproduce (for bugs)

Deploy generic cluster using kind (kind v0.17.0 go1.19.2 linux/amd64)
Setup openfaas using arkade install openfaas (Version: 0.8.56)
Deploy a function using instructions from, https://www.openfaas.com/blog/health-and-readiness-for-functions/#your-own-readiness-endpoint
Check for function availability (i.e. kubectl get deploy/readyornot -n openfaas-fn -w)

Context

I'm currently working on a masters capstone project which involves deploying cloud native applications using open source applications and comparing performance and cost against vendor specific products (i.e. AWS EKS, AWS Lambda). I'm hoping to publish my research eventually to highlight the benefits of open source solutions such as OpenFaas.

I also work for a company called Biodepot LLC which is also involved in developing solutions for bioinformatics problems using the cloud so my research will be handed over upon graduation.

Your Environment

Docker version docker version (e.g. Docker 17.0.05 ):


Client: Docker Engine - Community
Version:           20.10.23
API version:       1.41
Go version:        go1.18.10
Git commit:        7155243
Built:             Thu Jan 19 17:36:25 2023
OS/Arch:           linux/amd64
Context:           default
Experimental:      true

Server: Docker Engine - Community Engine: Version: 20.10.23 API version: 1.41 (minimum version 1.12) Go version: go1.18.10 Git commit: 6051f14 Built: Thu Jan 19 17:34:14 2023 OS/Arch: linux/amd64 Experimental: false containerd: Version: 1.6.15 GitCommit: 5b842e528e99d4d4c1686467debf2bd4b88ecd86 runc: Version: 1.1.4 GitCommit: v1.1.4-0-g5fd4c4d docker-init: Version: 0.19.0 GitCommit: de40ad0


* Are you using Kubernetes or faasd?
`kubernetes`

* Operating System and version (e.g. Linux, Windows, MacOS):

NAME="Ubuntu" VERSION="20.04.5 LTS (Focal Fossa)" ID=ubuntu ID_LIKE=debian PRETTY_NAME="Ubuntu 20.04.5 LTS" VERSION_ID="20.04" HOME_URL="https://www.ubuntu.com/" SUPPORT_URL="https://help.ubuntu.com/" BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" VERSION_CODENAME=focal UBUNTU_CODENAME=focal



* Link to your project or a code example to reproduce issue:

zhulik commented 1 year ago

alexellis commented 1 year ago

Hey @rgschmitz1

These features are working as intended in OpenFaaS Pro, and are not part of the free edition of OpenFaaS which you are using. OpenFaaS CE is a community project meant for hobbyists and experimentation.

Find out more about the differences at:

https://docs.openfaas.com/openfaas-pro/introduction/#workload-tuning

openfaas / of-watchdog