http beat consume > 2Gb of memory

[istrakhouski]

When there is no load on machine with httb beat every thing is ok, when we run load test, httpbeat begin consume more resources than our application. What is the reason if it still just should pull links and send responses to logstash?

Consumed memory:

Configuration: ` urls:

Each - URL endpoints to call. Below are the URL endpoint specific configurations

Url to monitor

  -
    url: http://localhost:80/service1/metrics
    method: get
    cron: '@every 15s'
    # Optional additional headers to send to the endpoint
    headers:
      Accept: application/json
    output_format: json
    json_dot_mode: replace

Url to monitor

  -
    url: http://localhost:80/service2/metrics
    method: get
    cron: '@every 15s'
    # Optional additional headers to send to the endpoint
    headers:
      Accept: application/json
    output_format: json
    json_dot_mode: replace

Url to monitor

  -
    url: http://localhost:80/service3/metrics
    method: get
    cron: '@every 15s'
    # Optional additional headers to send to the endpoint
    headers:
      Accept: application/json
    output_format: json
    json_dot_mode: replace

Url to monitor

  -
    url: http://localhost:80/service4/metrics
    method: get
    cron: '@every 15s'
    # Optional additional headers to send to the endpoint
    headers:
      Accept: application/json
    output_format: json
    json_dot_mode: replace

`

[christiangalsterer]

I haven't seen this yet. Which version of the plugin to you use and which OS?

[istrakhouski]

Http beat version: 3.3.0. OS: NAME="Amazon Linux AMI" VERSION="2016.09" ID="amzn" ID_LIKE="rhel fedora" VERSION_ID="2016.09" PRETTY_NAME="Amazon Linux AMI 2016.09" ANSI_COLOR="0;33" CPE_NAME="cpe:/o:amazon:linux:2016.09:ga" HOME_URL="http://aws.amazon.com/amazon-linux-ami/" Amazon Linux AMI release 2016.09

[stanmarsh2]

The problem is reproduced on 4.0.0 version.

[christiangalsterer]

I tried to reproduce it and under OSX I was not able to reproduce it. Will try to test in the next days in a linux machine

[christiangalsterer]

I finally made now a very simple first test with a new version on Ubuntu and didn't encounter the issue. RSS was around 10000. Will do some more tests.

Did you saw any differences with the new number of http endpoints you query and/or the frequency?

[istrakhouski]

Actually we wasn't able to reproduce it, we saw it couple of times, but didn't catch how and when it happens.