search

penberg / helix

Helix, a market data feed handler for C and C++.
Other
110 stars 35 forks source link

unordered_map & map usage #2

Closed ghost closed 8 years ago

ghost commented 9 years ago

i think unordered_map is rather slow thing. can we consider something else? like own fixed size hash map? also you want to NOT resize at all cost

penberg commented 9 years ago

@divaykin There's unordered_map::resize() that might help here. I'm open to using something else but we first need to come up with a benchmark to measure it.

ghost commented 9 years ago

see https://github.com/divaykin/hashmap-bench 42ns is non neglectable already.

i'd expect max number of orders in the book to be around 4-10k (to be verified) and ID to be incremented sequentially by the venue (not completely sure though).

of course we'd need a better hashmap but i'm trying to show that unordered_map is expensive.

ghost commented 9 years ago

we can experiment with simpler hash function for unordered_map but essentially even with mod it'd be slow

jvirtanen commented 9 years ago

A while ago I calculated the maximum number of concurrent price levels and the maximum number of orders per price level for all instruments trading on NASDAQ on a particular day. That might be interesting information regarding this project as well.

ghost commented 9 years ago

@jvirtanen that's very interesting to know indeed, thanks

ghost commented 9 years ago

according to perf report this line https://github.com/penberg/helix/blob/master/src/nasdaq/itch50_session.cc#L132 takes a lot of time, just lookup by uint16_t.

even though entire 14GB file was processed in about a minute

penberg commented 8 years ago

Fixed by 58546cde6ff59df2cf6f5761d6acb755fc7954db.