Drain the body of a response before closing it

[kostyantyn]

If body is read, connection will be returned to Transport.idleConn pool otherwise it will be just closed.

[joshuap]

@kostyantyn do you mind elaborating on this one a bit? I'm not sure I understand fully.

[kostyantyn]

sure, i'll make an example:

package main

import (
    "net/http"
    "net"
    "fmt"
    "time"
    "sync/atomic"
    "io/ioutil"
)

func main() {
    var num int64 = 0

    client := &http.Client{
        Transport: &http.Transport{
            MaxIdleConnsPerHost: 2,
            Dial: func(network, addr string) (net.Conn, error) {
                fmt.Println(atomic.AddInt64(&num, 1))
                d := net.Dialer{
                    KeepAlive: 5 * time.Second,
                }
                return d.Dial(network, addr)
            },
        },
    }

    ch := make(chan int, 2)
    for i := 0; i < 2; i++ {
        go func() {
            for i := 0; i < 5; i++ {
                res, _ := client.Get("https://github.com/")
                ioutil.ReadAll(res.Body)
                res.Body.Close()
            }
            ch <- 1
        }()
    }
    <-ch
    <-ch
    fmt.Println("DONE")
}

So, we start here 2 go-routines and each of them performs 5 calls to Github. At the end of each call we read the body and close it. We also track how many times the new socket connection is established. If we run this code, we will get:

➜  test  go run main.go
1
2
DONE  

We can see that we try to establish the connection to Github twice, once per each go-routine, which is obvious.

Now, let's comment two lines:

// "io/ioutil"
// ioutil.ReadAll(res.Body) 

and run again:

➜  test  go run main.go
1
2
3
4
5
6
7
8
9
10
DONE

here we can see that http client establishes the connection every time we perform the call. The reason is that connection is returned to the Transport.idleConn pool for further reuse only when the body is fully read, otherwise it remains active until it's terminated by timeout.

[joshuap]

Ah, got it. Thanks for the awesome example!