search

thulab / iot-benchmark

IoT-benchmark is a tool for benchmarking TSDB in IoT scenario.
Apache License 2.0
179 stars 101 forks source link

Low performance when running benchmark code on a different PC than database server #404

Open moaazassali opened 3 months ago

moaazassali commented 3 months ago

If I run the benchmark directly on the same server as the one with the database, the performance is good; I get an ingestion rate of 3.5M/s.

However, since the the benchmark uses a lot of the cpu, I decided to run the benchmark on another PC which connects to the same server with the database. But now, the ingestion rate is 2M/s.

Note that this other PC is more powerful than the server with the database. So, the speed at which it can generate the SQL commands is not an issue, and I would expect that the ingestion rate on the server should stay the same or increase since a lot of CPU usage is freed when the benchmark code is not running on the server directly.

By running htop on the server, I can see that the max network rx speed is around 25MBps, so it seems that the benchmark can't transmit the sql command batches fast enough.

The actual bandwidth between the PC and the server is also not an issue, I tested the speed with iperf and it is around 1Gbit = 125MBps.

This is the configuration I used in both tests:

BENCHMARK_WORK_MODE=testWithDefaultPath
OPERATION_PROPORTION=1:0:0:0:0:0:0:0:0:0:0
GROUP_NUMBER=10
DEVICE_NUMBER=1000
SENSOR_NUMBER=3
CLIENT_NUMBER=8
BATCH_SIZE_PER_WRITE=30000
POINT_STEP=200
LOOP=1
SpriCoder commented 3 months ago

I think it's better that you can provide more details about other PC and your server which can help me to get your points~ If network is not the bottle, I think we need to take a look of CPU