Is the Java template single-threaded?

[tuct]

TLDR: Java 8 function can not execute in parallel (multi threaded) in one container

We run git (jgit) code in the java 8 functions and sometimes this functions can take 2-5min to complete. Under high load (in our case multiple requests to one function) we saw logs like this in the function: Connection to XXX.XXX.XXX.XXXX:8081 reset by peer.

Took us a while to figure out that the reason for that is the Java 8 http Server which runs with the default executor which is single threaded. So when we call the functions multiple times the java side only processed one by one and this sometimes took longer than the tcp idle time of apline linux => Connection reset by peer.

For shorter running code this might not be an issue for quite some time, but still can happen under heavy load.

Expected Behaviour

Connection is not reset under this circumstances

Current Behaviour

Connection from of-watchdog to java 8 entrypoint get reset by java8

Possible Solution

a) switch to multi threaded executor in the template b) allow setting the executor from the function code

Steps to Reproduce (for bugs)

1) Create a java 8 function that idles for 5 min 2) Call it twice 3) connection reset by peer will show up in logs after 1-2 min

Context

Your Environment

• Docker version docker version (e.g. Docker 17.0.05 ):

• Are you using Docker Swarm or Kubernetes (FaaS-netes)?

• Operating System and version (e.g. Linux, Windows, MacOS):

• Link to your project or a code example to reproduce issue:

[MattMacGillivray]

interested in this, having a single container execute multiple functions would be handy.

[tuct]

@qmnonic I do that with the node10 template as it is using express.js with a slithly customized template. The main problem I currently have with the single threaded Java server is concurrent function execution is simple not possible: if I have one Java function that I call multiple times in a row, Java will process one call after each other, the node10 template would allow paraplell execution as it use express.js for serving the http call from the watchdog.

[alexellis]

Please feel free to experiment with the template or to build your own using Vert.x or whatever technology you prefer.

I would need some numbers and steps to reproduce the problem reported to confirm it.

I'm addition memory/CPU and scaling may be important to the solution.

[alexellis]

How did you confirm/observe that the built-in Sun server behaves that way?

[alexellis]

/set title: Is the Java template single-threaded?

[alexellis]

hey -c 50 -n 1000 http://192.168.0.35:8080/function/sun

Summary:
  Total:        0.9233 secs
  Slowest:      0.1834 secs
  Fastest:      0.0043 secs
  Average:      0.0420 secs
  Requests/sec: 1083.0591

  Total data:   13000 bytes
  Size/request: 13 bytes

Response time histogram:
  0.004 [1]     |
  0.022 [245]   |■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■
  0.040 [329]   |■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■
  0.058 [140]   |■■■■■■■■■■■■■■■■■
  0.076 [173]   |■■■■■■■■■■■■■■■■■■■■■
  0.094 [83]    |■■■■■■■■■■
  0.112 [22]    |■■■
  0.130 [4]     |
  0.148 [1]     |
  0.165 [0]     |
  0.183 [2]     |

Latency distribution:
  10% in 0.0159 secs
  25% in 0.0224 secs
  50% in 0.0332 secs
  75% in 0.0612 secs
  90% in 0.0776 secs
  95% in 0.0858 secs
  99% in 0.1084 secs

Details (average, fastest, slowest):
  DNS+dialup:   0.0005 secs, 0.0043 secs, 0.1834 secs
  DNS-lookup:   0.0000 secs, 0.0000 secs, 0.0000 secs
  req write:    0.0001 secs, 0.0000 secs, 0.0013 secs
  resp wait:    0.0412 secs, 0.0041 secs, 0.1832 secs
  resp read:    0.0002 secs, 0.0000 secs, 0.0012 secs

Status code distribution:
  [200] 1000 responses

This is a basic load-test with 50 concurrent threads / workers. It appears to have completed very quickly vs. using -c 1, but since I don't have your example, findings or steps to reproduce the problem, I can't really help more until I hear from you.

Can you confirm that you've read and are following along with this: https://docs.openfaas.com/architecture/performance/ ?

[alexellis]

Could you be hitting this?

Note about socket backlogs

When binding to an address and port number, the application can also specify an integer backlog parameter. This represents the maximum number of incoming TCP connections which the system will queue internally. Connections are queued while they are waiting to be accepted by the HttpServer. When the limit is reached, further connections may be rejected (or possibly ignored) by the underlying TCP implementation. Setting the right backlog value is a compromise between efficient resource usage in the TCP layer (not setting it too high) and allowing adequate throughput of incoming requests (not setting it too low).

From: https://docs.oracle.com/javase/8/docs/jre/api/net/httpserver/spec/com/sun/net/httpserver/HttpServer.html

Or perhaps this? https://github.com/openfaas/faas-netes/pull/524

[kameshchauhan]

@tuct Is the intention here to load test your function and assert the single/multi threaded behavior of the server? OR a feature request to have multi-threaded server?

[alexellis]

/lock