search

hibari / gdss-brick

http://hibari.github.com/hibari-doc/
Other
11 stars 2 forks source link

Refactor #state record in brick_ets #9

Open tatsuya6502 opened 11 years ago

tatsuya6502 commented 11 years ago

From Hibari Contributor's Guide - Draft https://github.com/hibari/hibari-doc/blob/master/src/hibari/hibari-contributor-guide.en.txt#L729

NOTE: The #state record is probably too big. Profiling suggests that a substantial amount of CPU time is being spent in erlang:setelement(); I'm guessing that's related to #state record updates, but I'm not 100% certain. There are about 43 members in that record, so refactoring by moving some items (e.g. operation counts like n_add) to the process dictionary or ETS is likely a good idea.

tatsuya6502 commented 10 years ago

From Hibari Contributor's Guide - Draft

NOTE: The #state record is probably too big. Profiling suggests that a substantial amount of CPU time is being spent in erlang:setelement(); I'm guessing that's related to #state record updates, but I'm not 100% certain. There are about 43 members in that record, so refactoring by moving some items (e.g. operation counts like n_add) to the process dictionary or ETS is likely a good idea.

This commit https://github.com/hibari/gdss-brick/commit/2e52fc5fc5a64 eliminated the following four fields from the #state record.

-record(state, {
    ...
          syncsum_count = 0 :: non_neg_integer(), % DEBUG?? syncsum stats
          syncsum_msec = 0 :: non_neg_integer(),
          syncsum_len = 0 :: integer(),
          syncsum_tref :: reference(),

In the commit, a folsom based metrics system (built around ETS) was introduced to provide more detailed statistics in production.

Will replace the operation counters like n_add with folsom soon. There are about 10 more fields to go.