Open Altonymous opened 11 years ago
Can you try using Go's http package (the code below should work).
I get similar errors when running on my mac. I can try running this benchmark on linux tomorrow.
As far as I know, unless you add a Connection: close
header, Go's http server will keep the connection open. My guess is that it's hitting the file descriptor limit.
package main
import (
"encoding/json"
"net/http"
)
// structs
type Reading struct {
Id string `json:"id"`
Name string `json:"name"`
}
func main() {
http.HandleFunc("/machines/", func(w http.ResponseWriter, r *http.Request) {
// Setup readings
readings := prepareReadings()
// return readings
w.Write([]byte(readingsToString(readings)))
})
http.ListenAndServe(":3000", nil)
}
func readingsToString(readings []Reading) string {
data, err := json.Marshal(readings)
if err != nil {
panic(err)
}
return string(data)
}
func prepareReadings() []Reading {
var readings []Reading
for i := 1; i <= 1; i++ {
readings = append(readings, Reading{Name: "Thing"})
}
return readings
}
It may be hitting the limit. That's why I included my settings. Unless its another setting I'm missing. If you are getting the same error using their libraries I'm sure ill get more of the same. If it is a core library issue do you want to open an issue since you will probably be able to explain it better?
I swapped over to running it on linux, and don't seem to have the same problems. It looks like I may have found an issue related to Mac's only. :(
Oh I forgot, I also updated to 1.1beta of golang. So it could be that as well.
This seems related:
Well I've moved all my stuff to ec2 to continue my testing and I'm running ubuntu there as well. I'm getting the error again.... :(
I used the code you generated above, installed rc3, and dropped it out onto my aws instance.
The problem still appears to be happening. I'm not sure how to determine what the cause of the issue is at this point.
Here are the limits for my process...
[api #3312 -- limits]
Limit Soft Limit Hard Limit Units
Max cpu time unlimited unlimited seconds
Max file size unlimited unlimited bytes
Max data size unlimited unlimited bytes
Max stack size 8388608 unlimited bytes
Max core file size 0 unlimited bytes
Max resident set unlimited unlimited bytes
Max processes 59479 59479 processes
Max open files 4999999 4999999 files
Max locked memory 65536 65536 bytes
Max address space unlimited unlimited bytes
Max file locks unlimited unlimited locks
Max pending signals 59479 59479 signals
Max msgqueue size 819200 819200 bytes
Max nice priority 0 0
Max realtime priority 0 0
Max realtime timeout unlimited unlimited us
When I 'lsof | wc -l', I dont' get above 1000. Generally in the ~800-850 range.
When I 'watch --interval=2 'netstat -tuna |grep "SYN_RECV"|wc -l'', I am generally in the ~130-250 range.
I'm not sure if this is a me, web.go, or go issue... but I was doing a little light load testing and discovered that under a little load I started getting Connection reset by peer errors.
Here's a gist of my setup: https://gist.github.com/Altonymous/7bab9ee8caa6da368883
Some things to note. the machine doing the siege is different than the machine serving the requests.
The limits setup on the machine serving the requests are as follows: $ launchctl limit cpu unlimited unlimited filesize unlimited unlimited data unlimited unlimited stack 8388608 67104768 core 0 unlimited rss unlimited unlimited memlock unlimited unlimited maxproc 1064 1064 maxfiles 10240 120000
$ ulimit -a core file size (blocks, -c) 0 data seg size (kbytes, -d) unlimited file size (blocks, -f) unlimited max locked memory (kbytes, -l) unlimited max memory size (kbytes, -m) unlimited open files (-n) 10240 pipe size (512 bytes, -p) 1 stack size (kbytes, -s) 8192 cpu time (seconds, -t) unlimited max user processes (-u) 1064 virtual memory (kbytes, -v) unlimited