Vegeta package getting connection reset by peer

[calvinvoo2]

Version and Runtime

Version: %!s(*bool=0xc42009e710)
Commit:
Go version:
Date:

Expected Behaviour

Expected behaviour is that we should see the same result, attacking using command line or using the go package.

So I did the load test using both the command line and using inside a Go Project When using command line vegeta attack -duration=5s -rate=150 -targets=targets.txt | vegeta report -reporter=text The results shows this Status Codes [code:count] 200:750

But when I put in on the go package, attacking the same API, it will get this error Get https://myapi.com?method=GET&proxy=true: read tcp 127.0.0.1:55057->127.0.0.1:443: read: connection reset by peer Get https://myapi.com?method=GET&proxy=true: read tcp 127.0.0.1:55063->127.0.0.1:443: read: connection reset by peer Get https://myapi.com?method=GET&proxy=true: read tcp 127.0.0.1:55064->127.0.0.1:443: read: connection reset by peer And most of the response will have 0 Status Codes

Steps to Reproduce

1. Create the Attack command line script like this one vegeta attack -duration=5s -rate=150 -targets=targets.txt | vegeta report -reporter=text

2. Create a Go Script that uses the Attack package and attack the same endpoint

3. The Go script result will have the status codes result like this map[0:106 200:144]

[tsenart]

Hello @calvinvoo2. Thanks for reporting this issue. It seems you're not using the latest version of Vegeta which should output something meaningful with vegeta -version. Please try that first.

In the Steps to Reproduce, please provide the Go code you are using. Thank you.

[calvinvoo2]

Sorry this is the version Version: v7.0.3 Commit: f5e8a53b900b1509b8719aa94c30d4cf087a77e9 Runtime: go1.10.2 darwin/amd64 Date: 2018-05-24T10:20:16Z+0100

For the steps to reproduce in the Go code is like this

            targeter := vegeta.NewStaticTargeter(vegeta.Target{
                Method: "GET",
                URL:    "MyAPI",
                // Header: header,
            })
            attacker := vegeta.NewAttacker(vegeta.Timeout(time.Duration(50) * time.Second))

            var metrics vegeta.Metrics
            for res := range attacker.Attack(targeter, rate, duration) {
                metrics.Add(res)
            }
            metrics.Close()
            fmt.Printf("Result (Total Request: %d):\n", metrics.Requests)
            fmt.Printf("Success rate: %.0f%% \r\n", metrics.Success*100)
            fmt.Printf("Status Codes[code:count]: %v\r\n\n", metrics.StatusCodes)

Is this enough?

[tsenart]

It is not enough. How can I reproduce your situation locally? Please provide everything needed for that :-)

[calvinvoo2]

package main

import (
    "encoding/json"
    "fmt"
    "time"

    vegeta "github.com/tsenart/vegeta/lib"
)

type ApiObject struct {
    URL  string `json:"url"`
    Name string `json:"name"`
}
type AttackData struct {
    API      []ApiObject `json:"api"`
    Auth     string      `json:"auth"`
    RPS      []int       `json:"rps"`
    Duration int         `json:"duration"`
}

func main() {
    jsonString := `{
        "duration": 2,
        "rps": [10,100,200,300],
        "api": [
            {
                "name": "Fetch List",
                "url": "{myownapi url}"
            },
        ],
    }`
    var attack AttackData
    err := json.Unmarshal([]byte(jsonString), &attack)
    if err != nil {
        fmt.Printf("ERR %v", err)
    }
    duration := time.Duration(attack.Duration) * time.Second

    for _, api := range attack.API {
        fmt.Printf("Attacking %s (%s)\r\n\n", api.Name, api.URL)
        for _, rps := range attack.RPS {
            fmt.Printf("With %d rps for %v\n", rps, duration)
            rate := uint64(rps)
            targeter := vegeta.NewStaticTargeter(vegeta.Target{
                Method: "GET",
                URL:    attack.API[0].URL,
            })
            attacker := vegeta.NewAttacker(vegeta.Timeout(time.Duration(50) * time.Second))

            var metrics vegeta.Metrics
            for res := range attacker.Attack(targeter, rate, duration) {
                metrics.Add(res)
            }
            metrics.Close()
            fmt.Printf("Result (Total Request: %d):\n", metrics.Requests)
            fmt.Printf("Success rate: %.0f%% \r\n", metrics.Success*100)
            fmt.Printf("Status Codes[code:count]: %v\r\n\n", metrics.StatusCodes)
        }
        fmt.Printf("\n==========================\n")
    }

}

The go script is like the above

[ghost]

Have you tried playing with the KeepAlive, Connections and Workers options? These are set in the CLI. The server you are testing is resetting the underlying TCP connections.

[tsenart]

@calvinvoo2: See above comment.

[calvinvoo2]

hi @ghost, I have tested the connections and workers options previously, is it to set it with higher numbers right? I will try the KeepAlive options and get back to you with the result.

[tsenart]

Yeah, just make sure the values are the same set via the CLI.

[tsenart]

Any developments?

[ryan-hixcare]

Get similar problems here,

jq -ncM '{
  method: "POST",
  url: "http://localhost:8080/v2.0/user/login",
  body: ({
    id: "admin",
    password: "adminpassword"
  } | @base64),
  header: {
    "Content-Type": ["application/json"]
  }
}' | vegeta attack -keepalive -format=json -duration=3s -max-connections=1000000 -rate=3000 -insecure -workers=30000 -connections=100000 | vegeta report -output="report.txt"

Looks like I get my connections, workers sufficient. But most of the request returns 0.

Requests      [total, rate, throughput]         9000, 3063.67, 109.35
Duration      [total, attack, wait]             22.077s, 2.938s, 19.139s
Latencies     [min, mean, 50, 90, 95, 99, max]  13.794ms, 5.327s, 5.306s, 7.323s, 8.067s, 10.185s, 21.849s
Bytes In      [total, mean]                     1182416, 131.38
Bytes Out     [total, mean]                     99218, 11.02
Success       [ratio]                           26.82%
Status Codes  [code:count]                      0:6389  200:2414  500:197  
Error Set:
Post "http://localhost:8080/v2.0/user/login": read tcp 127.0.0.1:52751->127.0.0.1:8080: read: connection reset by peer
...