Open epsylonix opened 3 years ago
Yes, it's the parser. Some profiling using cprof:
1> cprof:start(), R=eredis:q(E, ["HGETALL", "h"], 100000), cprof:pause(),R.
{ok,[<<"1">>,
<<"aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa"...>>]}
2> cprof:analyse(eredis_parser).
{eredis_parser,82216,
[{{eredis_parser,buffer_to_binary,1},6853},
{{eredis_parser,buffer_create,1},6853},
{{eredis_parser,get_newline_pos,1},6852},
{{eredis_parser,do_parse_multibulk,3},6852},
{{eredis_parser,parse_bulk,1},6851},
{{eredis_parser,do_parse_bulk,1},6851},
{{eredis_parser,buffer_size,1},6851},
{{eredis_parser,buffer_hd,1},6851},
{{eredis_parser,return_result,3},6850},
{{eredis_parser,parse,2},6850},
{{eredis_parser,parse_multibulk,2},6849},
{{eredis_parser,buffer_append,2},6849},
{{eredis_parser,parse_multibulk,1},2},
{{eredis_parser,init,0},1},
{{eredis_parser,do_parse_multibulk,2},1}]}
3> cprof:stop().
16107
4> f(R).
5> cprof:start(), R=eredis:q(E, ["HGET", "h", 1], 100000), cprof:pause(),R.
{ok,<<"aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa"...>>}
6> cprof:analyse(eredis_parser).
{eredis_parser,34257,
[{{eredis_parser,return_result,3},6850},
{{eredis_parser,parse,2},6850},
{{eredis_parser,parse_bulk,2},6849},
{{eredis_parser,buffer_size,1},6849},
{{eredis_parser,buffer_append,2},6849},
{{eredis_parser,parse_bulk,1},2},
{{eredis_parser,buffer_to_binary,1},2},
{{eredis_parser,buffer_create,1},2},
{{eredis_parser,init,0},1},
{{eredis_parser,get_newline_pos,1},1},
{{eredis_parser,do_parse_bulk,1},1},
{{eredis_parser,buffer_hd,1},1}]}
7> cprof:stop().
16107
From the documentation I understand that there is a performance optimization for multibulk response processing that sets eredis apart from other libraries. It seems that (some?) multibulk responses are actually processed orders of magnitude slower than non-multibulk responses. Here is a sample code to reproduce poor performance for multibulk response parsing:
The size of the returned data is almost the same but the first command resulted in a RESP Bulk String response and the second one returned a RESP Array (multibulk) response. The first command was executed in 58ms, the second took almost 17 seconds!