globalsign / mgo

The MongoDB driver for Go
Other
1.97k stars 231 forks source link

query concurrent case cpu 100% #321

Closed wzhsh90 closed 5 years ago

wzhsh90 commented 5 years ago

1、create global session dialInfo := &mgo.DialInfo{ Addrs: []string{host}, Database: gDb, Source: gDb, Timeout: 5 time.Second, ReadTimeout: 2 time.Second, Username: user, Direct: true, Password: password, MaxIdleTimeMS: 30000, PoolLimit: pool, FailFast: true, } s, err := mgo.DialWithInfo(dialInfo) if err != nil { beego.Info("create 创建mgo失败") beego.Info(err) } var endTime = time.Now().UnixNano() fmt.Println("连接所花时间:", (endTime-startTime)/1000000) s.SetMode(mgo.Monotonic, false) globalS = s

2、connect func connect(collection string) (mgo.Session, mgo.Collection) { //s := globalS.Copy() s := globalS.Clone() c := s.DB(gDb).C(collection) return s, c }

3、query func FindOne(collection string, query, result interface{}) error { ms, c := connect(collection) defer ms.Close() return c.Find(query).One(result) } 1、concurrency 500 requestsPerSecond 200 case my cpu to 100%, and I find mongo logs authenticate before every query, solve the problem?

domodwyer commented 5 years ago

Hi @wzhsh90

When there are no available connections in the connection pool, and the pool limit has not been reached, a new connection is created. This new connection performs authentication with the MongoDB instance (if required) and then caches the credentials for reuse - this operation is expensive by design (see RFC5802).

If you're suddenly making a large number of requests to MongoDB, mgo will attempt to grow the connection pool (up to the configured limit) in order to satisfy the concurrent load, and you'll be performing a SCRAM auth for each new connection concurrently - this will obviously have a CPU cost, especially if you're using a VM or a CPU without hardware acceleration for SHA hashing.

Once these connections are established, they are available for future use as needed. All of this is as designed, if you are benchmarking your code, or deploying it into a high-traffic environment, please warm your instances instead of suddenly loading them heavily.

Dom