hashicorp / faas-nomad

OpenFaaS plugin for Nomad
https://www.openfaas.com
MIT License
255 stars 45 forks source link

Auto Scaling is not working #89

Open mabuaisha opened 4 years ago

mabuaisha commented 4 years ago

What are the steps to reproduce this issue?

  1. Deploy OpenFaaS on Nomad cluster
  2. Deploy simple NodeJS (V12) function
version: 1.0
provider:
  name: openfaas
  # This can be override upon request/need
  gateway: http://192.168.0.17:8080
functions:
  fibfunction:
    lang: node12
    handler: ./fibfunction
    image: mabuaisha/fibfunction:latest
    labels:
      datacenters: "dc1"
      com.openfaas.scale.min: "3"
      com.openfaas.scale.max: "5"
      com.openfaas.scale.factor: "20"
    environment:
      read_timeout: 20s
      write_timeout: 20s
      write_debug: true

The above is the configuration for simple nodejs12 specified as the following

module.exports = async (event, context) => {
  let number = event['body'];
  number = number.replace(/\n/g, '');
  // console.log(event);
  // console.log(context);
  //
  //   if (!number || !isNumeric(number)) {
  //       return context
  //           .status(400)
  //           .fail(createErrorResponse("Please pass a valid number 'number'."));
  //   }
  //
  //   number = parseInt(number);
  //   const result = fib(number);

  return context
    .status(200)
    .succeed({'result': number})
};

What happens?

It never create 3 instances of the function based on the com.openfaas.scale.min: "3" and cannot see the auto scaling is working.

I'm testing ab -n 100000 -c 40 -p ab.data -T "content-type: text/plain" http://192.168.0.17:8080/function/fibfunction

But never scale automatically

What were you expecting to happen?

  1. Set the initial number of functions based on the value set on com.openfaas.scale.min=3
  2. Auto scaling automatically for heavy load

Any logs, error output, etc?

 curl -s http://192.168.0.17:8080/system/function/fibfunction | jq
{
  "name": "fibfunction",
  "image": "mabuaisha/fibfunction:latest",
  "invocationCount": 0,
  "replicas": 1,
  "envProcess": "",
  "availableReplicas": 1,
  "labels": {
    "com.openfaas.scale.factor": "20",
    "com.openfaas.scale.max": "5",
    "com.openfaas.scale.min": "3",
  },
  "annotations": null
}
curl -s http://192.168.0.17:8080/system/info | jq
{
  "provider": {
    "provider": "",
    "version": {
      "sha": "",
      "release": "0.3.6"
    },
    "orchestration": "nomad"
  },
  "version": {
    "commit_message": "Update swagger for missing secret definitions",
    "sha": "a65df4795bc66147c41161c48bfd4c72f60c7434",
    "release": "0.9.14"
  }
}

What versions of software are you using?

Operating System: Centos 7

OpenFaaS Version: … 0.9.14

Nomad Version: … 0.10.2

This is somehow similar to this issue https://github.com/hashicorp/faas-nomad/issues/61

@acornies your help on this much appreciated