Performance of del() is pretty bad

[staltz]

Context: if we're going to do many deletes (for the purpose of compacting the log), then we need deletes to happen quickly.

I ran a simple test that adds 1000 messages sequentially, and then deletes 500 messages (every odd-numbered msg), and the timing is:

• publish 1000: 2786.441ms
• delete 500: 19394.860ms

I'm studying the flame chart in Chrome devtools to figure out why, but wanted to inform that this is an area to work on.

Hypothesis: there are lots of things going on for deletes:

1. Query the keys leveldb to get the offset
2. Delete that offset in the log
3. Update the keys leveldb

[staltz]

If deletion indeed takes 38ms for each msg, then for a database in production (1 million messages) it would take 5h to delete half of all msgs. :scream: