Full table scan select is 10x slower than Lovefield

arthurhsu commented 4 years ago

If the performance benchmark is run only once, then the numbers are comparable.

The slow down is observed only when repetition is greater than 2.

arthurhsu commented 4 years ago

Observation of one sample run:

rep 1:   2.95    3.95    6.04   12.13   11.57
rep 2:  42.69   87.74  122.74  160.36  180.76
rep 3:  23.65   83.92  121.67  171.05  187.08
rep 4: 155.46  176.38  251.96  184.76  209.39
rep 5:  38.93   83.52  123.39  158.88  198.51

Variation is very significant. I wonder if this is related to V8 itself on Windows.

Attempted debugging so far:

Inspected that Global, Cache, and Index objects are properly destroyed and recreated in between repetitions.
Timed Cache.getMany() and indexed range scan, both are still single digits execution time.
Use ES5 module of Lovefield-ts instead of ES6, and performance numbers are similar.

arthurhsu commented 4 years ago

If I disabled the validation, then the first iteration becomes very slow too. If I enabled the validation for every iteration, then the select speed is significantly up.

Given the duration is fixed to time for test itself, the most plausible theory is that V8 is generating different code paths for validated iteration and non-validated iteration.

arthurhsu commented 4 years ago

Tested on Safari 13.1 and there's no such problem. Pretty sure it's a Chrome issue.

Numbers on Safari 13.1 (MacOS 10.15.4, 2.9GHz Intel Core i9, 32GB RAM)

rep1  8  13  17  22  25
rep2  5  11  15  45  23
rep3 12  11  13  18  28
rep4  7  25  13  20  31
rep5  8  10  15  24  24

arthurhsu / lovefield-ts

Full table scan select is 10x slower than Lovefield #18