search

openfaas / faas

OpenFaaS - Serverless Functions Made Simple
https://www.openfaas.com
MIT License
25.16k stars 1.94k forks source link

Proposal: Benchmarking performance and stability #34

Closed vielmetti closed 7 years ago

vielmetti commented 7 years ago

The benchmark tools at https://github.com/networknt/microservices-framework-benchmark provides a common "microservices" framework for testing and characterizing performance across systems; it would be useful to set something up for this.

The benchmark provided is a simple "Hello world" fetch, which is simple to implement.

Expected Behavior

Performance of FaaS is well-characterized.

Current Behavior

Performance of FaaS seems to be good, but there aren't numbers.

Possible Solution

Do some simple load testing, and characterize performance, prior to running a benchmark.

Context

The "microservices" list has a lot of interesting projects in it which is worthwhile to review in any case.

alexellis commented 7 years ago

I have done something similar with go bench - but since it runs in a way non-determinsitically it seemed hard to do repeat runs. Is this something you'd like to experiment with @vielmetti ?

vielmetti commented 7 years ago

I'm looking at ab (apache benchmark) if I can set it up correctly, since it seems to have some good repeated testing and measurement.

http://httpd.apache.org/docs/current/programs/ab.html

alexellis commented 7 years ago

By all means play with ab, but I've had mixed results with Docker swarm in the past due to the way it keeps connections alive etc.

alexellis commented 7 years ago

Please see - https://blog.alexellis.io/openfaas-serverless-acceleration/

Happy to re-open the issue if you find some more time for testing @vielmetti

I hear Gattling is really good for ramping up tests.