Closed russelldb closed 9 years ago
I thought this would bite us. Someone has a branch with hashdict stuff (from elixir). Sean will know about it, though maybe it was my branch. I haven't seen graphs/tables, so I don't know the comparison, but worth checking.
Those branches will be stale. I asked about HashDict, not ready for primetime. Hopefully this will do for now.
Wish you'd said earlier that you thought this was an issue.
Still todo
to/from_binary/2
to the riak_dt
behaviourto/from_binary/2
for other data types:+1: 44383b1
@borshop merge
After a report on IRC of Maps with 13k set elements inside taking up to 80seconds to merge, I ran some benchmarks comparing
dict
andorddict
(as I had a suspicion this was the bulk of problem.)This PR swaps orddict for dict. There may well be many more optimisations to do, but as the attached graph, and tables below show, this is a mighty big speed up.
This plot is insert times for dict vs orddict in an orswot.
Merge times, orswot, fully probably* fully disjoint sets with probably\ 20 actors orddict vs dict (times are microseconds)
[_Probably fully disjoint as crypto:rand_bytes(30) was used to generate elements, they could clash I guess. *_Probably 20 actors as actor chosen at random from a list of 20 randomly generated 8 byte actor ids, but if you have fewer than 20 elements, unlikely you have 20 actors, eh?]
This plot compares merge times for the code that used lots of simple set operations, vs. an uglier, single fold over the unioned keyspace of the map.