ysrc / yulong-hids-archived

[archived] 一款实验性质的主机入侵检测系统
Other
2.16k stars 581 forks source link

fixbug: server.cleanThread无sleep导致mongo server负载过大 #15

Closed chuanbowen closed 6 years ago

ihacku commented 6 years ago

我补充一下背景 “mongo进程被kill N小时之后重启,发现server进程和mongo进程打满了CPU。”

neargle commented 6 years ago

感谢pr,可是我没搞懂为啥要在这里加。 这里ip数过多还会 sleep 一下, mongodb连不上的情况不应该会报 error 吗?

chuanbowen commented 6 years ago

@neargle 感谢回复,一下是我对代码的理解: cleanThread中,代码逻辑: for { 查询离线72小时机器; if 离线机器数 > 100: sleep 1 minute }

如果没有离线机器,那就是会持续query mongodb。

image

neargle commented 6 years ago

写了个脚本验证了一下:

package main

import (
    "log"
    "time"

    mgo "gopkg.in/mgo.v2"
    "gopkg.in/mgo.v2/bson"
)

var DB, _ = conn("127.0.0.1:27017", "agent")

func cleanThread() {
    client := DB.C("client")
    for {
        var offlineIPList []string
        log.Println(bson.M{"uptime": bson.M{"$lte": time.Now().Add(time.Hour * time.Duration(-72))}})
        err := client.Find(
            bson.M{"uptime": bson.M{"$lte": time.Now().Add(time.Hour * time.Duration(-72))}}).Distinct("ip", &offlineIPList)
        if err != nil {
            log.Println(err.Error())
        }
        if len(offlineIPList) >= 100 {
            time.Sleep(time.Second * 60)
            continue
        }
        for _, ip := range offlineIPList {
            log.Println(bson.M{"uptime": bson.M{"$lte": time.Now().Add(time.Hour * time.Duration(-72))}})
            err = DB.C("client").Remove(bson.M{"ip": ip})
            if err != nil {
                log.Println(err.Error())
            }
        }
    }
}

func conn(netloc string, dbname string) (*mgo.Database, error) {
    session, err := mgo.Dial(netloc)
    if err != nil {
        return nil, err
    }
    session.SetMode(mgo.Monotonic, true)
    db := session.DB(dbname)
    return db, nil
}

func main() {
    cleanThread()
}

虽然没有复现问题,但是这里确实需要sleep,感谢pr。