Try to use Jemalloc 5.3.0

[kenhys]

Just evaluate 5.3.0, will not be merged.

pushed on #522

[ashie]

Thanks, I also want to evaluate it. One more thing, please consider disabling MADV_FREE https://github.com/fluent/fluentd-docker-image/pull/350

[ashie]

I want to apply it if we observe no noticeable difference in memory usage with the following procedure: https://github.com/fluent/fluentd/issues/1657#issue-248177661

[ashie]

I want to apply it if we observe no noticeable difference in memory usage with the following procedure:

I've evaluated it by this way.

no jemalloc

1976192 874144 286404 /opt/fluent/bin/ruby /usr/sbin/td-agent --no-supervisor -c Projects/Fluentd/fluentd/issues/1657/test-in.conf
1976230 524128 182512 /opt/fluent/bin/ruby /usr/sbin/td-agent --no-supervisor -c Projects/Fluentd/fluentd/issues/1657/test-out.conf

jemalloc 3.6

1979520 421888 277680 /opt/fluent/bin/ruby /usr/sbin/td-agent --no-supervisor -c Projects/Fluentd/fluentd/issues/1657/test-in.conf
1979540 305524 136140 /opt/fluent/bin/ruby /usr/sbin/td-agent --no-supervisor -c Projects/Fluentd/fluentd/issues/1657/test-out.conf

jemalloc 5.3 no-madv-free

2037935 473352 282064 /opt/fluent/bin/ruby /usr/sbin/td-agent --no-supervisor -c Projects/Fluentd/fluentd/issues/1657/test-in.conf
2037963 433920 172116 /opt/fluent/bin/ruby /usr/sbin/td-agent --no-supervisor -c Projects/Fluentd/fluentd/issues/1657/test-out.conf

jemalloc 5.3

2097286 473292 273052 /opt/fluent/bin/ruby /usr/sbin/td-agent --no-supervisor -c Projects/Fluentd/fluentd/issues/1657/test-in.conf
2097307 401472 163032 /opt/fluent/bin/ruby /usr/sbin/td-agent --no-supervisor -c Projects/Fluentd/fluentd/issues/1657/test-out.conf

• Still jemalloc 3 seems less memory usage than others.
• no-madv-free doesn't seem effective in this context.

[kenhys]

visualized the result.

[ashie]

BTW the config names test-in.conf and test-out.conf are inverted (comes from the first comment of the issue).

• test-in.conf: forwarder
• test-out.conf: aggregator

[kenhys]

It seems that it is reasonable to stay with jemalloc 3.x in point view of performance for a while. It does not mean that declining migration to jemalloc 5.x in the future.