search

fireproof-storage / fireproof

Realtime database, runs anywhere. Install Fireproof in your front-end app or edge function, and sync data via any backend.
https://fireproof.storage
Other
219 stars 16 forks source link

perf discrepency between benchmarks #77

Closed jchris closed 5 months ago

jchris commented 5 months ago

There is a new benchmark here: https://github.com/fireproof-storage/fireproof/blob/45508c265aa290275a7a9cde2c26f685325c4963/packages/fireproof/test/fireproof.test.js#L131

Here is output from inserting 3000 records in 13 seconds:

  benchmarking a database
insert and read 3000 records: 13.106s
begin compact
COMPACT: 7.956s
compacted reopen again: 6.898s
compacted allDocs new DB2: 1.718s
compacted get all new DB2: 11.249s
compacted changes new DB2: 1.696s
compacted newDb2 insert and read 100 records: 711.663ms
    ✔ insert and read many records (43371ms)

  1 passing (43s)

Here is another benchmark tool output for inserting 1000 records in almost 6 minutes!

The tool: https://github.com/dstanesc/fireproof-playground/pull/1

fetch: 512.111ms
Fetched 1000 entries
Database initial state 
Database allDocs 0
all docs: 0.323ms
log: 0 - 0
batch0: 611.751ms
log: 100 - 2
batch100: 2.828s
log: 200 - 4
batch200: 6.941s
log: 300 - 6
batch300: 13.300s
log: 400 - 8
batch400: 21.084s
log: 500 - 10
batch500: 31.257s
log: 600 - 12
batch600: 43.199s
log: 700 - 14
batch700: 58.026s
log: 800 - 16
batch800: 1:13.833 (m:ss.mmm)
log: 900 - 18
batch900: 1:31.854 (m:ss.mmm)
store: 5:42.954 (m:ss.mmm)

I think the workloads should be close enough to the same, the next step is to do a version of the fast bencmark that does 100 item batches and see if that's comparably slow.

jchris commented 5 months ago

ok it's figured out. fix coming

jchris commented 5 months ago

fix will be these types

jchris commented 5 months ago

https://github.com/fireproof-storage/fireproof/pull/67

jchris commented 5 months ago

here is the bigger file outputing in 15 seconds (4000 records). Fix will come in 0.17

fetch: 477.246ms
Fetched 4000 entries
Database initial state 
Database allDocs 0
all docs: 0.74ms
log: 0 - 0
batch0: 477.796ms
log: 500 - 2
batch500: 712.422ms
log: 1000 - 4
batch1000: 1.011s
log: 1500 - 6
batch1500: 1.467s
log: 2000 - 8
batch2000: 2.017s
log: 2500 - 10
batch2500: 2.720s
log: 3000 - 12
batch3000: 3.423s
log: 3500 - 14
batch3500: 3.998s
store: 15.841s
query: 297.805ms
Query results 3
jchris commented 5 months ago

Fireproof 0.17.1 takes it down to less than 4 seconds. When I started optimizing in 0.16 it was taking HOURS. Thanks for the benchmark!

fetch: 451.733ms
Fetched 4000 entries
Database initial state 
Database allDocs 0
all docs: 0.405ms
log: 0 - 0
batch0: 157.843ms
log: 500 - 2
batch500: 222.098ms
log: 1000 - 4
batch1000: 310.638ms
log: 1500 - 6
batch1500: 401.283ms
log: 2000 - 8
batch2000: 509.162ms
log: 2500 - 10
batch2500: 623.778ms
log: 3000 - 12
batch3000: 722.557ms
log: 3500 - 14
batch3500: 815.652ms
store: 3.774s
query: 244.394ms
Query results 3