search

yabeda-rb / yabeda-prometheus

Adapter to expose metrics collected by Yabeda plugins to Prometheus
MIT License
113 stars 18 forks source link

Errno::ENOMEM: Cannot allocate memory #12

Open berniechiu opened 4 years ago

berniechiu commented 4 years ago

Hi Yabeda-rb team,

We've bumped into issues like this

Screenshot 2020-04-14 17 55 37

Is there any possibility to dump the file store properly?

Envek commented 4 years ago

Hi! Can you please provide some more information?

  1. What are versions of yabeda, yabeda-prometheus, and prometheus-client?
  2. How is your prometheus-client is configured?
  3. How many metrics do you have?
  4. When you observe memory issues? E.g. long uptime or peak load or something else?

Anything that can help me to understand why it happens.

berniechiu commented 4 years ago

What are versions of yabeda, yabeda-prometheus, and prometheus-client?

The latest version as of today

    yabeda (0.5.0)
      concurrent-ruby
      dry-initializer
    yabeda-prometheus (0.5.0)
      prometheus-client (~> 1.0)
      yabeda (~> 0.5)
    prometheus-client (1.0.0)

How is your prometheus-client is configured?

We've only added this line as the documentation suggested

Prometheus::Client.config.data_store = Prometheus::Client::DataStores::DirectFileStore.new(dir: '/tmp/prometheus_direct_file_store')

How many metrics do you have?

We've only installed these two

    yabeda-puma-plugin (0.3.0)
      json
      puma
      yabeda (~> 0.2)
    yabeda-rails (0.4.0)
      rails
      yabeda (~> 0.4)

When you observe memory issues? E.g. long uptime or peak load or something else?

It's randomly popping up in our Error Monitoring actually, not particularly under high load.

Thanks a lot!

berniechiu commented 4 years ago

I've also posted the issue here, might be because of the ruby client https://github.com/prometheus/client_ruby/issues/185