performance test BCDB [development_bcdb2]

despiegk commented 4 years ago

[x] remove the current performance test code, not needed, bloated
[x] add a simple test which uses j.tools.timer to test some basic performance items

remarks

don't use any new constructs, testing frameworks, ...
just use our existing test baseclasses

AhmedHanafy725 commented 4 years ago

A smiple test is added here, but it needs to the fixes has been done on development_fixes branch to be run.

xmonader commented 4 years ago

I created development_bcdb2 branch from development_bcdb and merged development_fixes in. I'll move it to verification for now and to be verified on development_bcdb2

AhmedHanafy725 commented 4 years ago

I'll remove this from verification as the test fails in bcdb.reset() in case of writing on sqlite backend, and @Hamdy is already working on this in #340

john-kheir commented 4 years ago

@xmonader or @despiegk please verify this ticket once it is done.. just to make sure it is what you guys asked for

AhmedHanafy725 commented 4 years ago

The test has been moved to development_fixes branch, and it works fine now. commit: f2380846a9c0fabe62f05eb90780a5f1d84d10c5

How to run:

kosmos -p "j.data.bcdb.test('performance')"

xmonader commented 4 years ago

Thanks @AhmedHanafy725

AhmedHanafy725 commented 4 years ago

Location

here

Result

Write objects of 64 Bytes of string (zdb backend)
duration:4.825684070587158
nritems:1000.0
performance:207/sec

Write indexed objects of 64 Bytes of string (zdb backend)
duration:4.246560335159302
nritems:1000.0
performance:235/sec

Query objects (zdb backend)
duration:0.1674649715423584
nritems:1000.0
performance:5971/sec

Write objects of 64 Bytes of string (rdb backend)
duration:5.198923110961914
nritems:1000.0
performance:192/sec

Write indexed objects of 64 Bytes of string (rdb backend)
duration:4.367642402648926
nritems:1000.0
performance:228/sec

Query objects (rdb backend)
duration:0.1694011688232422
nritems:1000.0
performance:5903/sec

Write objects of 64 Bytes of string (sqlite backend)
duration:23.900813817977905
nritems:1000.0
performance:41/sec

Write indexed objects of 64 Bytes of string (sqlite backend)
duration:18.164596796035767
nritems:1000.0
performance:55/sec

Query objects (sqlite backend)
duration:0.17668652534484863
nritems:1000.0
performance:5659/sec

Write objects of 64 Bytes of string (sonic)
duration:5.737967491149902
nritems:1000.0
performance:174/sec

Query in sonic
duration:1.1330623626708984
nritems:1000.0
performance:882/sec

threefoldtecharchive / jumpscaleX_core