httpuv endpoint cannot handle nagios' check_http and apache benchmark (ab) calls.

[zimnyx]

Hi! By default, connection is not closed on server side during normal processing, you can see it with simples endpoint: grappa@adel /tmp/httpuv/demo $ curl -v http://127.0.0.1:9876

• Rebuilt URL to: http://127.0.0.1:9876/
• Trying 127.0.0.1...
• Connected to 127.0.0.1 (127.0.0.1) port 9876 (#0)

  GET / HTTP/1.1 Host: 127.0.0.1:9876 User-Agent: curl/7.47.0 Accept: /

  < HTTP/1.1 200 OK < Content-Type: text/html < Content-Length: 3 <

• Connection #0 to host 127.0.0.1 left intact (see last line).

There is no option to force closing connection at the end of response. It could be done by adding "Connection: close" header support in case of HTTP/1.1

[jcheng5]

Good idea. As a workaround, you can add the header to the response yourself.

Just curious, what are you using httpuv for?

[zimnyx]

Hi Joe, thanks for interest! I renamed this ticket, because it may be not related to disconnecting issue.

Following your suggestion, I just tried sending "Connection: close" header from server, but it didn't fixed issue with both ab and /usr/lib/nagios/plugins/check_http. These are standard tools very widely used.

So here's what I observed (httpuv 1.3.3):

Here's the endpoint:

library(httpuv)

app <- list(
  call = function(req) {
    list(
      status = 200L,
      headers = list(
        'Content-Type' = 'text/html'
      ),
      body = "foo"
    )
  } 
)

runServer("0.0.0.0", 9876, app, 250)

Let's try apache benchmark

$ ab http://127.0.0.1:9876/
This is ApacheBench, Version 2.3 <$Revision: 1706008 $>
Benchmarking 127.0.0.1 (be patient)...apr_pollset_poll: The timeout specified has expired (70007)

Let's try nagios check

grappa@adel /tmp/httpuv $ /usr/lib/nagios/plugins/check_http -H localhost -p 9876
CRITICAL - Socket timeout after 10 seconds

Let's try curl

$ curl -v http://127.0.0.1:9876
* Rebuilt URL to: http://127.0.0.1:9876/
*   Trying 127.0.0.1...
* Connected to 127.0.0.1 (127.0.0.1) port 9876 (#0)
> GET / HTTP/1.1
> Host: 127.0.0.1:9876
> User-Agent: curl/7.47.0
> Accept: */* 
> 
< HTTP/1.1 200 OK
< Content-Type: text/html
< Content-Length: 3
< 
* Connection #0 to host 127.0.0.1 left intact
foo

So looks like only curl can communicate with httpuv. ab and check_http are waiting for something. I tried to debug src/http.cpp but didn't find a reason.

P.S. My app provides two endpoints using httpuv. One is for data science stuff and other is a simple GET healthcheck.

[zimnyx]

If I can help somehow, please let me know.

[jcheng5]

Hmmm. Can you capture the headers with ab and check_http?

[zimnyx]

Sorry for delay, here's the ab output in verbose mode

This is ApacheBench, Version 2.3 <$Revision: 1706008 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking 127.0.0.1 (be patient)...INFO: GET header == 
---
GET / HTTP/1.0
Host: 127.0.0.1:9876
User-Agent: ApacheBench/2.3
Accept: */*

---
LOG: header received:
HTTP/1.1 200 OK
Content-Type: text/html
Content-Length: 3

LOG: Response code = 200
apr_pollset_poll: The timeout specified has expired (70007)

[zimnyx]

I can prepare a patch, if you would give me just a hint, what is causing timeout.

[jcheng5]

I think the fix would need to be, to detect HTTP/1.0 requests and, if no connection header is provided, close the socket right after the response is sent. IIRC, technically HTTP/1.0 isn't supposed to support keepalive at all, so especially in the face of no connection header, closing is the only thing that's allowed.

httpuv doesn't respect this because it wasn't written for HTTP 1.0 at all--I originally wrote it to support interactive web applications, and didn't concern myself with HTTP 1.0 spec compliance (you can see that the response even says HTTP/1.1). However, I certainly would not be opposed if you wanted to implement the correct behavior.

[wch]

Here's an article that explains how exactly this should work: https://reinir.github.io/articles/http-slim-and-apachebench.html

According to the article, if a server receives a HTTP 1.0 request, it can (and should) send back a 1.1 response. However, it should add Connection: close.

[wch]

To get apachebench to work, you can add a Connection: keep-alive header, and use ab -k.

app <- list(
  call = function(req) {
    list(
      status = 200L,
      headers = list(
        'Content-Type' = 'text/html',
        Connection = 'Keep-Alive'
      ),
      body = "abc"
    )
  } 
)

runServer("0.0.0.0", 5000, app)

The output from ab (I'm running in a Docker container, hence the weird command line and IP address):

$ docker run --rm jordi/ab ab -k -n 1 -c 1 -v 2 http://10.0.0.53:5000/
This is ApacheBench, Version 2.3 <$Revision: 1796539 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking 10.0.0.53 (be patient)...INFO: GET header == 
---
GET / HTTP/1.0
Connection: Keep-Alive
Host: 10.0.0.53:5000
User-Agent: ApacheBench/2.3
Accept: */*

---
LOG: header received:
HTTP/1.1 200 OK
Content-Type: text/html
Connection: keep-alive
Content-Length: 3

..done

Server Software:        
Server Hostname:        10.0.0.53
Server Port:            5000

Document Path:          /
Document Length:        3 bytes

Concurrency Level:      1
Time taken for tests:   0.011 seconds
Complete requests:      1
Failed requests:        0
Keep-Alive requests:    1
Total transferred:      90 bytes
HTML transferred:       3 bytes
Requests per second:    93.69 [#/sec] (mean)
Time per request:       10.673 [ms] (mean)
Time per request:       10.673 [ms] (mean, across all concurrent requests)
Transfer rate:          8.23 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        1    1   0.0      1       1
Processing:    10   10   0.0     10      10
Waiting:        9    9   0.0      9       9
Total:         11   11   0.0     11      11