[CHORE] Encountered some problems implementing verificationQuery

[Subsegment]

Description

I have implemented verificationQuery, but when I run the benchmark, I find some differences between the data I read from the database and the data that benmark verifies. So I added some code snippets for debug.I'm now commenting out those lines of code because it's not required for formal code

https://github.com/Subsegment/iot-benchmark/blob/2b705cbcaef03866490f80b6bf678526050522cf/cnosdb/src/main/java/cn/edu/tsinghua/iot/benchmark/cnosdb/CnosDB.java#L48-L49 https://github.com/Subsegment/iot-benchmark/blob/2b705cbcaef03866490f80b6bf678526050522cf/cnosdb/src/main/java/cn/edu/tsinghua/iot/benchmark/cnosdb/CnosDB.java#L120-L142

I looked up the timestamp of the data that didn't pass benchmark and found these two in the written file

g_0,device=d_4 s_0=true,s_1=6i,s_2=781i,s_3=0i,s_4=0i,s_5=781.54,s_6=6.63,s_7=0.0,s_8="ZM",s_9="Kl" 1640608895592000000 g_0,device=d_4 s_0=true,s_1=6i,s_2=781i,s_3=0i,s_4=0i,s_5=781.54,s_6=6.65,s_7=0.0,s_8="HE",s_9="cy" 1640608895592000000

According to the data written in sequence, the later data should prevail, so the data queried from the CnosDB database is the second one, but benmark used the first one when verifying the data. Is there a problem in this case?

Control Variable Method

I tried setting DB_SWITCH to InfluxDB at the same time and running it using the test code in influxdb directory, and found that the lineprotocol data is written to the file in the same order. This means that using influxdb for verificationQuery can present the same problem, but I did not test it

Config File

generate

DB_SWITCH=CnosDB

HOST=127.0.0.1

PORT=8902

DB_NAME=test

BENCHMARK_WORK_MODE=generateDataMode
FILE_PATH=/tmp/data/test
DEVICE_NUMBER=5
SENSOR_NUMBER=10
CLIENT_NUMBER=5
BATCH_SIZE_PER_WRITE=10
OPERATION_PROPORTION=1:0:0:0:0:0:0:0:0:0:0
BIG_BATCH_SIZE=100

write

DB_SWITCH=CnosDB

HOST=127.0.0.1

PORT=8902

DB_NAME=test
BENCHMARK_WORK_MODE=verificationWriteMode

FILE_PATH=/tmp/data/test
DEVICE_NUMBER=5
SENSOR_NUMBER=10
CLIENT_NUMBER=5
BATCH_SIZE_PER_WRITE=10
OPERATION_PROPORTION=1:0:0:0:0:0:0:0:0:0:0
BIG_BATCH_SIZE=100
IS_COPY_MODE=true

query

DB_SWITCH=CnosDB

HOST=127.0.0.1

PORT=8902

DB_NAME=test

BENCHMARK_WORK_MODE=verificationQueryMode

FILE_PATH=/tmp/data/test
DEVICE_NUMBER=5
SENSOR_NUMBER=10
CLIENT_NUMBER=5
BATCH_SIZE_PER_WRITE=10
OPERATION_PROPORTION=1:0:0:0:0:0:0:0:0:0:0
BIG_BATCH_SIZE=100
IS_COPY_MODE=true

Debug

The data of CnosDB return

The data of verificationQuery record

Additional

I would like to know if there is something wrong with my code implementation or my configuration file is wrong. If my question is not very clear, I hope you can ask and add

[SpriCoder]

Get it ~ I will take a look

[SpriCoder]

Oh, I get your point. From my perspective, it because the overlap of timestamp. More specific benchmark generate two same timestamp which may cause overlap. We always use verification mode in sequence mode(IS_OUT_OF_ORDER=false), but in default config IS_OUT_OF_ORDER is true which may cause this problem. So I think it's a good idea to add config check when load configuration to forbidden unsequence mode. What's your opinion? Maybe you have a better solution