search

yahoojapan / ngtd

Serving NGT over HTTP or gRPC ※This project is not maintained. We have moved to a new product, [Vald](https://vald.vdaas.org) .
Apache License 2.0
38 stars 10 forks source link

Benchmark Code #25

Closed kpango closed 4 years ago

cloverrose commented 5 years ago

I'd like to try this job.

I think it is easy to use ngtd insert api for setup index, but ngtd can't start up without NGT index. So, at first we should fix this problem. I'll make this issue with detail info.

cloverrose commented 5 years ago

Oh, I'm sorry this problem does not happen now (and may did not happen) I can create index from scratch by only use ngtd HTTP API.

cloverrose commented 5 years ago

On the first phase I try to add search (or searchbyid) benchmark code.

For it I think it is useful to add dummy db which does not use any real db to ignore db specific performance differences or change. Do you have any concern about it?

kpango commented 5 years ago

As a precaution, I think that it is necessary to make the vector data as unbalanced as possible so as not to bias the vector data as a benchmark.

cloverrose commented 5 years ago

I plan to use JMeter for benchmark (and it is not run in CI), is it right? Or did you mean write BenchmarkXXX function in xxx_test.go?

kpango commented 5 years ago

I mean you should write BenchmarkXXX function code in xxx_benchmark_test.go. After that you should write e2e test code using JMeter or Vegeta or ApacheBench. I strongly recommend create e2e test package inside the ngtd using vegeta sdk.

cloverrose commented 5 years ago

Oh I see. I'm a Go newbie, so it may takes time.

kpango commented 5 years ago

It's okay, I'm proficient in Golang. I can review your code.

cloverrose commented 5 years ago

Thank you!! 😄

kpango commented 5 years ago

Although this is a digression, I'm already headed to the next version of ngtd (clusterize & native core api). If you have no time right now, you can cooperate after publishing the next version.