search

autodeployai / ai-serving

Serving AI/ML models in the open standard formats PMML and ONNX with both HTTP (REST API) and gRPC endpoints
Apache License 2.0
144 stars 31 forks source link

单台服务器4核8G xgboost 访问在100QPS以下 返回在10ms左右 达到500QPS 就有超过500ms响应 时长 #10

Open adangadang opened 2 years ago

adangadang commented 2 years ago

单台服务器4核8G xgboost 访问在100QPS以下 cpu占10% 返回在10ms左右 达到500QPS cpu占30% 就有超过500ms响应 时长 json 请求

这个怎么优化

scorebot commented 2 years ago

@adangadang I suppose you are using the REST API, if so you can config the following part: https://github.com/autodeployai/ai-serving/blob/master/src/main/resources/application.conf#L17

  ai-predicting-dispatcher {
    type = Dispatcher
    executor = "thread-pool-executor"
    thread-pool-executor {
      fixed-pool-size = 16
    }
    throughput = 1
  }

About the dispatcher, see the AKKA doc: https://doc.akka.io/docs/akka/current/typed/dispatchers.html

adangadang commented 2 years ago

直接 调用你docker 镜像