Puma with yabeda-sidekiq use the same port

[vk26]

Hi! I have rails app(with puma) and sidekiq in one machine. I setup yabeda for rails and sidekiq using manual.

# Gemfile:
gem 'yabeda-prometheus'
gem 'yabeda-sidekiq'
gem 'yabeda-rails'

# config/initializers/sidekiq.rb:
Sidekiq.configure_server do |_config|
  Yabeda::Prometheus::Exporter.start_metrics_server!
end

# config.ru:
use Yabeda::Prometheus::Exporter

When I start puma server and sidekiq I get error from sidekiq:

Address already in use - bind(2) for 0.0.0.0:3000 (Errno::EADDRINUSE)

If I try get metrics localhost:3000/metrics It doesn't show sidekiq-metrics, only rails-metrics. Is it normal behavior? And I should start puma and sidekiq in different ports in this case? For ex:

PORT=8080 bundle exec rails s
PORT=3000 bundle exec sidekiq

And setup prometheus to fetch metrics from 8080 and 3000 ports separatly.

[Envek]

It doesn't show sidekiq-metrics, only rails-metrics. Is it normal behavior?

Yes, if you use official prometheus ruby client in default mode. If you need to collect metrics from multiple processes on the same machine/container and expose on single endpoint then either use yabeda-prometheus-mmap gem or look at Multi-process server support section in yabeda-prometheus gem README.

For both yabeda-prometheus and yabeda-prometheus-mmap gems port is configured with following environment variables:

Port is configured by PROMETHEUS_EXPORTER_PORT or PORT variables (default is 9394).

Also process managers like overmind automatically sets different values of PORT for every process to fix that problem.

[vk26]

@Envek thank you for feedback :+1: I tried to use gem yabeda-prometheus-mmap and use the same port(9394) for puma and sidekiq, but I faced with the same problem.

PROMETHEUS_EXPORTER_PORT=9394 rails s
PROMETHEUS_EXPORTER_PORT=9394 sidekiq

After strart sidekiq I get Address already in use - bind(2) for 0.0.0.0:9394 (Errno::EADDRINUSE) and localhost:9394/metrics returns only rails/puma metrics without sidekiq.

# Gemfile
gem 'yabeda-prometheus-mmap'
gem 'yabeda-sidekiq'
gem 'yabeda-puma-plugin'
gem 'yabeda-rails'

# config.ru
require 'yabeda/prometheus/mmap'
use Yabeda::Prometheus::Exporter

# config/initializers/sidekiq.rb
require 'yabeda/prometheus/mmap'
Sidekiq.configure_server do |_config|
  Yabeda::Prometheus::Exporter.start_metrics_server!
end

# config/puma.rb
activate_control_app
plugin :yabeda
plugin :yabeda_prometheus

It seems works with only puma or only sidekiq, but isn't together.

[Envek]

With yabeda-prometheus-mmap you don't need to start exporter from Sidekiq process on the same machine.

This is usually needed if you run sidekiq on separate machine or container.

[vk26]

I try remove code from config/initializers/sidekiq.rb, but after i don't get sidekiq-metrics. I don't understand about p.2 in README gem yabeda-prometheus-mmap:

2. Run web-server from long-running processes (delayed jobs, …):
require 'yabeda/prometheus/mmap'
Yabeda::Prometheus::Exporter.start_metrics_server!

Is it not about start metrics export (ex, for sidekiq)? Where should this code be located?

[Envek]

Oh, sorry, I messed up official prometheus client direct file store and prometheus-mmap.

yabeda-prometheus-mmap helps you collect and expose metrics from forking multi-process servers (like Unicorn or Puma in cluster mode). With it you still need to launch exporters in every independent processes (puma, sidekiq, etc) on non-conflicting ports.

Multi-process server support in yabeda-prometheus gem also (as implementation side-effect) allows you to collect metrics from independent processes. With it you can avoid launching web server from Sidekiq processes on the same machine. However, there were some caveats with this mode (too many files created or similar).

[vk26]

@Envek thank you for explanations! :+1: