Very large buffer_size can cause young gc issues

[ppf2]

Have a use case where the buffer_size setting is set to 500Mb using aws-plugin 2.4.0. This appears to cause frequent and long young gcs with the young gen filling up quickly (and in turn cause other performance issues in the cluster). Would be nice to add a resiliency limit and/or specify a warning/disclaimer in the documentation.

[2014-12-11T00:29:56,255Z]  [WARN ]  [elasticsearch[node_name][scheduler][T#1]]  [org.elasticsearch.monitor.jvm]  [node_name] [gc][young][181102][7206] duration [2.9s], collections [1]/[2.9s], total [2.9s]/[28.1m], memory [21.6gb]->[22.1gb]/[30.1gb], all_pools {[young] [509mb]->[505.9mb]/[865.3mb]}{[survivor] [5.5mb]->[7.8mb]/[108.1mb]}{[old] [21.1gb]->[21.6gb]/[29.2gb]}
[2014-12-11T00:29:59,793Z]  [WARN ]  [elasticsearch[node_name][scheduler][T#1]]  [org.elasticsearch.monitor.jvm]  [node_name] [gc][young][181103][7207] duration [3.1s], collections [1]/[3.5s], total [3.1s]/[28.2m], memory [22.1gb]->[22.1gb]/[30.1gb], all_pools {[young] [505.9mb]->[500mb]/[865.3mb]}{[survivor] [7.8mb]->[6.1mb]/[108.1mb]}{[old] [21.6gb]->[21.6gb]/[29.2gb]}
[2014-12-11T00:30:03,362Z]  [WARN ]  [elasticsearch[node_name][scheduler][T#1]]  [org.elasticsearch.monitor.jvm]  [node_name] [gc][young][181104][7208] duration [3.4s], collections [1]/[3.5s], total [3.4s]/[28.2m], memory [22.1gb]->[22.1gb]/[30.1gb], all_pools {[young] [500mb]->[502.4mb]/[865.3mb]}{[survivor] [6.1mb]->[7.4mb]/[108.1mb]}{[old] [21.6gb]->[21.6gb]/[29.2gb]}

[Leaf-Lin]

Does anyone know if cloud-aws can also cause long old gc like this:

[gc][old][12050098][3136] duration [1.1h], collections [209]/[1.2h], total [1.1h]/[2.7h], memory [27.2gb]->[28gb]/[29.8gb], all_pools {[young] [519.5mb]->[559.2mb]/[865.3mb]}{[survivor] [108.1mb]->[108.1mb]/[108.1mb]}{[old] [26.6gb]->[27.4gb]/[28.9gb]}