DataIntellectTech / kdb-chronicle-queue

Java adaptor from Chronicle queue to kdb+
3 stars 1 forks source link

Added new modes for improved performance testing #4

Closed jonnypress closed 3 years ago

jonnypress commented 3 years ago

Added two new modes for performance testing

-nocheck will not validate any data. It will just prepend a time column to the data, and then insert it into a table in memory. -discard will simply count the inbound data and then insert it, so the only cost we will be paying is serialisation

Use these to test performance after all the other testing has been done (i.e. the inbound data is correct)

Some basic testing. Output of \ts is (time in millis; space in bytes), lower time in millis is better

jonnyp@homer:/home/jonnyp/git/kdb-chronicle-queue/kdb$ q dummytp.q KDB+ 4.0 2020.06.18 Copyright (C) 1993-2020 Kx Systems

q)p:enlist each (.z.p;a;10f;299f;a) q)\ts:100000 0".u.upd[`trade;p]" 1842 6819376 q)\

jonnyp@homer:/home/jonnyp/git/kdb-chronicle-queue/kdb$ q dummytp.q -nocheck 1 KDB+ 4.0 2020.06.18 Copyright (C) 1993-2020 Kx Systems

q)p:enlist each (.z.p;a;10f;299f;a) q)\ts:100000 0".u.upd[`trade;p]" 288 6817616 q)\

jonnyp@homer:/home/jonnyp/git/kdb-chronicle-queue/kdb$ q dummytp.q -discard 1 KDB+ 4.0 2020.06.18 Copyright (C) 1993-2020 Kx Systems

q)p:enlist each (.z.p;a;10f;299f;a) q)\ts:100000 0".u.upd[`trade;p]" 186 1360