The on_load function for module prometheus_process_collector returned :error

[cbarratt]

[info] GET /metrics
[warn] The on_load function for module prometheus_process_collector returned {:error,
 {:load_failed,
  'Failed to load NIF library: \'dlopen(/Users/cba28/projects/superaggro/_build/dev/lib/prometheus_process_collector/priv/prometheus_process_collector.so, 2): no suitable image found.  Did find:\n\t/Users/cba28/projects/superaggro/_build/dev/lib/prometheus_process_collector/priv/prometheus_process_collector.so: unknown file type, first eight bytes: 0x7F 0x45 0x4C 0x46 0x02 0x01 0x01 0x00\''}}

[warn] The on_load function for module prometheus_process_collector returned {:error,
 {:load_failed,
  'Failed to load NIF library: \'dlopen(/Users/cba28/projects/superaggro/_build/dev/lib/prometheus_process_collector/priv/prometheus_process_collector.so, 2): no suitable image found.  Did find:\n\t/Users/cba28/projects/superaggro/_build/dev/lib/prometheus_process_collector/priv/prometheus_process_collector.so: unknown file type, first eight bytes: 0x7F 0x45 0x4C 0x46 0x02 0x01 0x01 0x00\''}}

[error] #PID<0.457.0> running Superaggro.Endpoint terminated
Server: 10.23.195.203:4000 (http)
Request: GET /metrics
** (exit) an exception was raised:
    ** (UndefinedFunctionError) function :prometheus_process_collector.collect_mf/2 is undefined (module :prometheus_process_collector is not available)
        (prometheus_process_collector) :prometheus_process_collector.collect_mf(:default, #Function<1.49806622/1 in :prometheus_text_format.registry_collect_callback/3>)
        (prometheus) /Users/cba28/projects/superaggro/deps/prometheus/src/prometheus_collector.erl:157: :prometheus_collector.collect_mf/3
        (prometheus) /Users/cba28/projects/superaggro/deps/prometheus/src/prometheus_registry.erl:58: :prometheus_registry."-collect/2-lc$^0/1-0-"/3
        (prometheus) /Users/cba28/projects/superaggro/deps/prometheus/src/prometheus_registry.erl:58: :prometheus_registry.collect/2
        (prometheus) /Users/cba28/projects/superaggro/deps/prometheus/src/formats/prometheus_text_format.erl:74: :prometheus_text_format.format/1
        (prometheus) /Users/cba28/projects/superaggro/deps/prometheus/src/metrics/prometheus_summary.erl:231: :prometheus_summary.observe_duration/4
        (superaggro) lib/prometheus_exporter.ex:2: Superaggro.PrometheusExporter.scrape_data/0
        (superaggro) lib/prometheus_exporter.ex:2: Superaggro.PrometheusExporter.call/2
        (superaggro) lib/superaggro/endpoint.ex:1: Superaggro.Endpoint.phoenix_pipeline/1
        (superaggro) lib/plug/debugger.ex:123: Superaggro.Endpoint."call (overridable 3)"/2
        (superaggro) lib/superaggro/endpoint.ex:1: Superaggro.Endpoint.call/2
        (plug) lib/plug/adapters/cowboy/handler.ex:15: Plug.Adapters.Cowboy.Handler.upgrade/4
        (cowboy) src/cowboy_protocol.erl:442: :cowboy_protocol.execute/4

This is from an Elixir 1.3.2 application, calling Prometheus.Registry.register_collector(:prometheus_process_collector) inside the application start function definition.

[deadtrickster]

Unfortunately Windows support isn't there yet (but it's coming, stay tuned!). If you run Linux on your servers you can use :os.type to register collector only when running on Linux:

iex(1)> :os.type
{:unix, :linux}

[cbarratt]

Hi @deadtrickster - I'm running this locally from macOS Sierra, is that also unsupported?

iex(1)> :os.type
{:unix, :darwin}

[deadtrickster]

Yeah not yet :-) It basically parses procfs stuff. Contributions are welcome ;-)

[brust]

Does not prometheus_process_collector work at MacOS?

[cschiewek]

I get the same error running in a docker container using elixir/latest image on osx.

[deadtrickster]

Guys, readme explicitly says this lib is linux-only currently. Sorry about that. But you can deploy on Linux. @cschiewek could you please tell me more about your elixir/latest usage? Do you use it as a base for deployment images?

[deadtrickster]

Works on FreeBSD now, probably should be working on MacOS. I don't have it so can't test.

[hawksight]

Tried to have a look but unfortunately getting a build error on macosx:

iex(apples@127.0.0.1)1> 15:21:51.657 request_id=ft8b6or2ia98kj0qovnsbjuelm9n65d3 [info] GET /metrics
15:21:51.670 [warn] The on_load function for module prometheus_process_collector returned:
{:error,
 {:load_failed,
  'Failed to load NIF library: \'dlopen(/Users/pfiddes/Projects/apples/_build/prod/lib/prometheus_process_collector/priv/prometheus_process_collector.so, 2): image not found\''}}

15:21:51.672 [error] #PID<0.410.0> running ApplesWeb.Endpoint terminated
Server: localhost:8101 (http)
Request: GET /metrics
** (exit) an exception was raised:
    ** (UndefinedFunctionError) function :prometheus_process_collector.collect_mf/2 is undefined (module :prometheus_process_collector is not available)
        (prometheus_process_collector) :prometheus_process_collector.collect_mf(:default, #Function<1.30220082/1 in :prometheus_text_format.registry_collect_callback/3>)
        (prometheus) /Users/pfiddes/Projects/apples/deps/prometheus/src/prometheus_collector.erl:169: :prometheus_collector.collect_mf/3
        (prometheus) /Users/pfiddes/Projects/apples/deps/prometheus/src/prometheus_registry.erl:78: :prometheus_registry."-collect/2-lc$^0/1-0-"/3
        (prometheus) /Users/pfiddes/Projects/apples/deps/prometheus/src/prometheus_registry.erl:78: :prometheus_registry.collect/2
        (prometheus) /Users/pfiddes/Projects/apples/deps/prometheus/src/formats/prometheus_text_format.erl:74: :prometheus_text_format.format/1
        (prometheus) /Users/pfiddes/Projects/apples/deps/prometheus/src/metrics/prometheus_summary.erl:257: :prometheus_summary.observe_duration/4
        (apples) lib/apples/prometheus.ex:19: Apples.PrometheusExporter.scrape_data/1
        (apples) lib/apples/prometheus.ex:19: Apples.PrometheusExporter.call/2

And on a docker container that previously worked (linux with elixir and phoenix deps installed):

15:51:37.043 request_id=pv4q4fsh4k02me870cvgpckun0de1q9n [info] GET /metrics
apples_1      | 15:51:37.050 [warn] The on_load function for module prometheus_process_collector returned {:error,
apples_1      |  {:load_failed,
apples_1      |   'Failed to load NIF library: \'Error relocating /app/_build/prod/lib/prometheus_process_collector/priv/prometheus_process_collector.so: __sprintf_chk: symbol not found\''}}
apples_1      |
apples_1      | 15:51:37.051 [error] #PID<0.574.0> running ApplesWeb.Endpoint terminated
apples_1      | Server: apples:80 (http)
apples_1      | Request: GET /metrics
apples_1      | ** (exit) an exception was raised:
apples_1      |     ** (UndefinedFunctionError) function :prometheus_process_collector.collect_mf/2 is undefined (module :prometheus_process_collector is not available)
apples_1      |         (prometheus_process_collector) :prometheus_process_collector.collect_mf(:default, #Function<1.12788421/1 in :prometheus_protobuf_format.registry_collect_callback/3>)
apples_1      |         (prometheus) /app/deps/prometheus/src/prometheus_collector.erl:169: :prometheus_collector.collect_mf/3
apples_1      |         (prometheus) /app/deps/prometheus/src/prometheus_registry.erl:78: :prometheus_registry."-collect/2-lc$^0/1-0-"/3
apples_1      |         (prometheus) /app/deps/prometheus/src/prometheus_registry.erl:78: :prometheus_registry.collect/2
apples_1      |         (prometheus) /app/deps/prometheus/src/formats/prometheus_protobuf_format.erl:45: :prometheus_protobuf_format.format/1
apples_1      |         (prometheus) /app/deps/prometheus/src/metrics/prometheus_summary.erl:257: :prometheus_summary.observe_duration/4
apples_1      |         (apples) lib/apples/prometheus.ex:19: Apples.PrometheusExporter.scrape_data/1
apples_1      |         (apples) lib/apples/prometheus.ex:19: Apples.PrometheusExporter.call/2

(ignore the apples_1 -> just docker-compose output)

I may have something wrong in my set-up, I did some funny stuff so that prometheous deps would (should) only build on mix.env ==prod (to get around mac issue).

Although I look in the container and I can see the file I'm being warned about as present:

/app # ls -ltr /app/_build/prod/lib/prometheus_process_collector/priv/
total 16
-rwxr-xr-x    1 root     root         15656 Oct 23 15:24 prometheus_process_collector.so
/app # ls -ltr /app/_build/prod/lib/prometheus_process_collector/priv/prometheus_process_collector.so
-rwxr-xr-x    1 root     root         15656 Oct 23 15:24 /app/_build/prod/lib/prometheus_process_collector/priv/prometheus_process_collector.so

Which makes me assume I've got something wrong in here (mix.exs):

defmodule Apples.Mixfile do
  use Mix.Project

  def project do
    [
      app: :apples,
      version: "0.0.1",
      elixir: "~> 1.5",
      elixirc_paths: elixirc_paths(Mix.env),
      compilers: [:phoenix, :gettext] ++ Mix.compilers,
      start_permanent: Mix.env == :prod,
      aliases: aliases(),
      deps: deps(Mix.env)
    ]
  end

  # Configuration for the OTP application.
  #
  # Type `mix help compile.app` for more information.
  def application() do
    [
      mod: {Apples.Application, []},
      extra_applications: extra_applications(Mix.env)
    ]
  end

  defp extra_applications(:prod) do
    extra_applications(:everything) ++ [
      :prometheus_ex,
      :prometheus_ecto, 
      :prometheus_phoenix,
      :prometheus_plugs, 
      :prometheus_process_collector
    ]
  end

  defp extra_applications(_) do
    [
      :logger,
      :runtime_tools,
    ]
  end

  # Specifies which paths to compile per environment.
  defp elixirc_paths(:test), do: ["lib", "test/support"]
  defp elixirc_paths(_),     do: ["lib"]

  # Specifies your project dependencies.
  #
  # Type `mix help deps` for examples and options.
  defp deps(_) do
    [
      {:phoenix, "~> 1.3.0"},
      {:phoenix_pubsub, "~> 1.0"},
      {:phoenix_ecto, "~> 3.2"},
      {:postgrex, ">= 0.0.0"},
      {:phoenix_html, "~> 2.10"},
      {:phoenix_live_reload, "~> 1.1.3", only: :dev},
      {:gettext, "~> 0.11"},
      {:cowboy, "~> 1.0"},
      {:prometheus_ex, "~> 1.0"},
      {:prometheus_plugs, "~> 1.0"},
      {:prometheus_ecto, "~> 1.0"},
      {:prometheus_phoenix, "~> 1.0"},
      {:prometheus_process_collector, "~> 1.0"},
    ]
  end

  # Aliases are shortcuts or tasks specific to the current project.
  # For example, to create, migrate and run the seeds file at once:
  #
  #     $ mix ecto.setup
  #
  # See the documentation for `Mix` for more info on aliases.
  defp aliases do
    [
      "ecto.setup": ["ecto.create", "ecto.migrate", "run priv/repo/seeds.exs"],
      "ecto.reset": ["ecto.drop", "ecto.setup"],
      "test": ["ecto.create --quiet", "ecto.migrate", "test"]
    ]
  end
end

Tried to built at specifically 1.0 for 'prometheus_process_collector' but it still goes with latest 1.21.

Also the following may be relates, got warning on elixir_make:

** (Mix) Could not compile dependency :elixir_make, "/root/.mix/rebar3 bare compile --paths "/app/_build/test/lib/*/ebin"" command failed. You can recompile this dependency with "mix deps.compile elixir_make", update it with "mix deps.update elixir_make" or clean it with "mix deps.clean elixir_make"

dep from process_coillector:

├── prometheus_process_collector ~> 1.2.1 (Hex package)
│   ├── elixir_make ~> 0.4 (Hex package)
│   └── prometheus ~> 3.4 (Hex package)

this fixed it for me, incase anyone else gets the same (follow what the error says basically):

mix deps.update elixir_make

[hawksight]

Actually just to confirm, I've definitely gone back on the dependency:

{:prometheus_process_collector, "1.1.0"}

Full List

  defp deps(_) do
    [
      {:phoenix, "~> 1.3.0"},
      {:phoenix_pubsub, "~> 1.0"},
      {:phoenix_ecto, "~> 3.2"},
      {:postgrex, ">= 0.0.0"},
      {:phoenix_html, "~> 2.10"},
      {:phoenix_live_reload, "~> 1.1.3", only: :dev},
      {:gettext, "~> 0.11"},
      {:cowboy, "~> 1.0"},
      {:prometheus_ex, "~> 1.0"},
      {:prometheus_plugs, "~> 1.0"},
      {:prometheus_ecto, "~> 1.0"},
      {:prometheus_phoenix, "~> 1.0"},
      {:prometheus_process_collector, "1.1.0"},
    ]
  end

I'm now working again:

apples_1      | 16:16:47.543 request_id=dco9jocgj7je4bjsdgui1spdaf0r3ccu [info] GET /metrics
apples_1      | 16:16:47.561 request_id=dco9jocgj7je4bjsdgui1spdaf0r3ccu [info] Sent 200 in 17ms

Once i get this working reliably I plan to put this demo project on github for others to use, but its essentially a blank phoenix app with the prometheus deps... that im using to test before integrating to our applications.

[deadtrickster]

Hi, unfortunately I pushed 1.2.1 with object files. Again (sigh). I don't even remember OS I built them on (I suspect FreeBSD), no wonder you have unknown symbol.

Updated to 1.2.2. Please try it!

Personally I use https://github.com/deadtrickster/slackin.ex as prom/plugs/phoenix playground. Maybe you'll find it useful as well.

Thanks for the report!

[deadtrickster]

I just realized slackin_ex needs tokens to work. heh. BTW @artukrap hacking phoenix dashboard here: https://github.com/deadtrickster/beam-dashboards/pull/8

[hawksight]

thanks @deadtrickster - nice to know it wasn't me being silly. Will take a look at that project.

So it seems to build locally on macosx now, however I am getting this warning:

===> Compiling prometheus_process_collector
src/prometheus_process_collector.erl:23: Warning: behaviour prometheus_collector undefined

When I'm running locally, can't get the /metrics endpoint. I think it's probably not starting up, not sure if it needs starting before my application?

Also it no longer builds in my docker container (linux, alpine 3.6.2) at mix compile stage:

===> Compiling prometheus_process_collector
make: Entering directory '/app/deps/prometheus_process_collector/c_src'
cc -O3 -std=c99 -finline-functions -Wall -Wmissing-prototypes -fPIC -I /usr/lib/erlang/erts-8.3/include/ -I /usr/lib/erlang/lib/erl_interface-3.9.3/include  -c -o /app/deps/prometheus_process_collector/c_src/prometheus_process_collector_nif.o /app/deps/prometheus_process_collector/c_src/prometheus_process_collector_nif.c
In file included from /app/deps/prometheus_process_collector/c_src/prometheus_process_collector_nif.c:5:0:
/app/deps/prometheus_process_collector/c_src/prometheus_process_collector.h:8:24: fatal error: sys/sysctl.h: No such file or directory
 #include <sys/sysctl.h>
                        ^
compilation terminated.
make: *** [Makefile:67: /app/deps/prometheus_process_collector/c_src/prometheus_process_collector_nif.o] Error 1
make: Leaving directory '/app/deps/prometheus_process_collector/c_src'
===> Hook for compile failed!

==> apples
** (Mix) Could not compile dependency :prometheus_process_collector, "/root/.mix/rebar3 bare compile --paths "/app/_build/test/lib/*/ebin"" command failed. You can recompile this dependency with "mix deps.compile prometheus_process_collector", update it with "mix deps.update prometheus_process_collector" or clean it with "mix deps.clean prometheus_process_collector"

Tried the obvious fixes of clean, update and compile, unfortunately not that simple of a fix though.

[deadtrickster]

so we are marking progress :-). can you please try master now? and share your alpine docker file if possible Thanks!

[hawksight]

Ok so on mac failing at the mix test (when it's compiling deps):

==> prometheus_process_collector
cc -O3 -std=c99 -arch x86_64 -finline-functions -Wall -Wmissing-prototypes -fPIC -I /usr/local/Cellar/erlang/20.1.2/lib/erlang/erts-9.1.2/include/ -I /usr/local/Cellar/erlang/20.1.2/lib/erlang/lib/erl_interface-3.10/include  -c -o /Users/pfiddes/Projects/apples/deps/prometheus_process_collector/c_src/prometheus_process_collector_backend.o /Users/pfiddes/Projects/apples/deps/prometheus_process_collector/c_src/prometheus_process_collector_backend.c
/Users/pfiddes/Projects/apples/deps/prometheus_process_collector/c_src/prometheus_process_collector_backend.c:19:1: warning: control reaches end of non-void function [-Wreturn-type]
}
^
/Users/pfiddes/Projects/apples/deps/prometheus_process_collector/c_src/prometheus_process_collector_backend.c:244:29: warning: implicit declaration of function 'kinfo_getproc' is invalid in C99 [-Wimplicit-function-declaration]
  struct kinfo_proc *proc = kinfo_getproc(pid);
                            ^
/Users/pfiddes/Projects/apples/deps/prometheus_process_collector/c_src/prometheus_process_collector_backend.c:244:22: warning: incompatible integer to pointer conversion initializing 'struct kinfo_proc *' with an expression of type 'int' [-Wint-conversion]
  struct kinfo_proc *proc = kinfo_getproc(pid);
                     ^      ~~~~~~~~~~~~~~~~~~
/Users/pfiddes/Projects/apples/deps/prometheus_process_collector/c_src/prometheus_process_collector_backend.c:253:6: warning: implicit declaration of function 'get_process_open_fd_totals' is invalid in C99 [-Wimplicit-function-declaration]
  if(get_process_open_fd_totals(pid, &pids_total)) {
     ^
/Users/pfiddes/Projects/apples/deps/prometheus_process_collector/c_src/prometheus_process_collector_backend.c:260:6: warning: implicit declaration of function 'get_process_limit' is invalid in C99 [-Wimplicit-function-declaration]
  if(get_process_limit(pid, RLIMIT_NOFILE, &rlimit)) {
     ^
/Users/pfiddes/Projects/apples/deps/prometheus_process_collector/c_src/prometheus_process_collector_backend.c:267:53: error: incomplete definition of type 'struct kinfo_proc'
  prometheus_process_info->start_time_seconds = proc->ki_start.tv_sec;
                                                ~~~~^
/Users/pfiddes/Projects/apples/deps/prometheus_process_collector/c_src/prometheus_process_collector_backend.c:244:10: note: forward declaration of 'struct kinfo_proc'
  struct kinfo_proc *proc = kinfo_getproc(pid);
         ^
/Users/pfiddes/Projects/apples/deps/prometheus_process_collector/c_src/prometheus_process_collector_backend.c:270:3: warning: implicit declaration of function 'time' is invalid in C99 [-Wimplicit-function-declaration]
  time(&now);
  ^
/Users/pfiddes/Projects/apples/deps/prometheus_process_collector/c_src/prometheus_process_collector_backend.c:271:55: error: incomplete definition of type 'struct kinfo_proc'
  prometheus_process_info->uptime_seconds = now - proc->ki_start.tv_sec;
                                                  ~~~~^
/Users/pfiddes/Projects/apples/deps/prometheus_process_collector/c_src/prometheus_process_collector_backend.c:244:10: note: forward declaration of 'struct kinfo_proc'
  struct kinfo_proc *proc = kinfo_getproc(pid);
         ^
/Users/pfiddes/Projects/apples/deps/prometheus_process_collector/c_src/prometheus_process_collector_backend.c:273:48: error: incomplete definition of type 'struct kinfo_proc'
  prometheus_process_info->threads_total = proc->ki_numthreads;
                                           ~~~~^
/Users/pfiddes/Projects/apples/deps/prometheus_process_collector/c_src/prometheus_process_collector_backend.c:244:10: note: forward declaration of 'struct kinfo_proc'
  struct kinfo_proc *proc = kinfo_getproc(pid);
         ^
/Users/pfiddes/Projects/apples/deps/prometheus_process_collector/c_src/prometheus_process_collector_backend.c:275:43: error: incomplete definition of type 'struct kinfo_proc'
  prometheus_process_info->vm_bytes = proc->ki_size;
                                      ~~~~^
/Users/pfiddes/Projects/apples/deps/prometheus_process_collector/c_src/prometheus_process_collector_backend.c:244:10: note: forward declaration of 'struct kinfo_proc'
  struct kinfo_proc *proc = kinfo_getproc(pid);
         ^
/Users/pfiddes/Projects/apples/deps/prometheus_process_collector/c_src/prometheus_process_collector_backend.c:277:43: error: incomplete definition of type 'struct kinfo_proc'
  prometheus_process_info->rm_bytes = proc->ki_rssize * pagesize();
                                      ~~~~^
/Users/pfiddes/Projects/apples/deps/prometheus_process_collector/c_src/prometheus_process_collector_backend.c:244:10: note: forward declaration of 'struct kinfo_proc'
  struct kinfo_proc *proc = kinfo_getproc(pid);
         ^
/Users/pfiddes/Projects/apples/deps/prometheus_process_collector/c_src/prometheus_process_collector_backend.c:279:30: error: incomplete definition of type 'struct kinfo_proc'
  struct rusage rusage = proc->ki_rusage;
                         ~~~~^
/Users/pfiddes/Projects/apples/deps/prometheus_process_collector/c_src/prometheus_process_collector_backend.c:244:10: note: forward declaration of 'struct kinfo_proc'
  struct kinfo_proc *proc = kinfo_getproc(pid);
         ^
6 warnings and 6 errors generated.
make[1]: *** [/Users/pfiddes/Projects/apples/deps/prometheus_process_collector/c_src/prometheus_process_collector_backend.o] Error 1
could not compile dependency :prometheus_process_collector, "mix compile" failed. You can recompile this dependency with "mix deps.compile prometheus_process_collector", update it with "mix deps.update prometheus_process_collector" or clean it with "mix deps.clean prometheus_process_collector"
==> apples
** (Mix) Could not compile with "make" (exit status: 2).
Depending on your OS, make sure to follow these instructions:

  * Mac OS X: You need to have gcc and make installed. Try running the
    commands "gcc --version" and / or "make --version". If these programs
    are not installed, you will be prompted to install them.

  * Linux: You need to have gcc and make installed. If you are using
    Ubuntu or any other Debian-based system, install the packages
    "build-essential". Also install "erlang-dev" package if not
    included in your Erlang/OTP version. If you're on Fedora, run
    "dnf group install 'Development Tools'".

FYI, Osx Sierra - 10.12.6

➜  apples git:(process-collector-update) ✗ gcc --version
Configured with: --prefix=/Library/Developer/CommandLineTools/usr --with-gxx-include-dir=/usr/include/c++/4.2.1
Apple LLVM version 9.0.0 (clang-900.0.38)
Target: x86_64-apple-darwin16.7.0
Thread model: posix
InstalledDir: /Library/Developer/CommandLineTools/usr/bin
➜  apples git:(process-collector-update) ✗ make --version
GNU Make 3.81
Copyright (C) 2006  Free Software Foundation, Inc.
This is free software; see the source for copying conditions.
There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A
PARTICULAR PURPOSE.

This program built for i386-apple-darwin11.3.0

alpine

dockerfile is part of an entire build system, but essentially for test stage, we run off a precompiled base with the relevant build tools etc.

Simplified out something like the following (note not sure the test run here when i run locally, but compile seems to work)

FROM elixir:1.5.2-alpine

WORKDIR /app

ENV MIX_ENV="test"

RUN apk --update add build-base git && rm -rf /var/cache/apk/*

RUN mix local.hex --force && mix local.rebar --force

ADD mix.exs /app
ADD mix.lock /app

RUN mix deps.get

ADD config /app/config
ADD lib /app/lib
ADD priv /app/priv
ADD test /app/test
ADD Makefile /app

RUN mix compile

CMD ["make","test"]

seems to run through ok but noticed a couple of potential warnings on compile:

=> prometheus_process_collector
cc -O3 -std=c99 -finline-functions -Wall -Wmissing-prototypes -fPIC -I /usr/local/lib/erlang/erts-9.1.2/include/ -I /usr/local/lib/erlang/lib/erl_interface-3.10/include  -c -o /app/deps/prometheus_process_collector/c_src/prometheus_process_collector_nif.o /app/deps/prometheus_process_collector/c_src/prometheus_process_collector_nif.c
cc -O3 -std=c99 -finline-functions -Wall -Wmissing-prototypes -fPIC -I /usr/local/lib/erlang/erts-9.1.2/include/ -I /usr/local/lib/erlang/lib/erl_interface-3.10/include  -c -o /app/deps/prometheus_process_collector/c_src/prometheus_process_collector_backend.o /app/deps/prometheus_process_collector/c_src/prometheus_process_collector_backend.c
cc /app/deps/prometheus_process_collector/c_src/prometheus_process_collector_nif.o /app/deps/prometheus_process_collector/c_src/prometheus_process_collector_backend.o -shared -L /usr/local/lib/erlang/lib/erl_interface-3.10/lib -lerl_interface -lei -o /app/deps/prometheus_process_collector/c_src/../priv/prometheus_process_collector.so

[deadtrickster]

Pushed one more commit, I hope this enough for Mac. (essentially enabling headers from freebsd)

What warnings on Alpine?

[hawksight]

Down from 6 to 1 error on mac:

==> prometheus_process_collector
cc -O3 -std=c99 -arch x86_64 -finline-functions -lutil -Wall -Wmissing-prototypes -fPIC -I /usr/local/Cellar/erlang/20.1.2/lib/erlang/erts-9.1.2/include/ -I /usr/local/Cellar/erlang/20.1.2/lib/erlang/lib/erl_interface-3.10/include  -c -o /Users/pfiddes/Projects/apples/deps/prometheus_process_collector/c_src/prometheus_process_collector_backend.o /Users/pfiddes/Projects/apples/deps/prometheus_process_collector/c_src/prometheus_process_collector_backend.c
clang: warning: -lutil: 'linker' input unused [-Wunused-command-line-argument]
In file included from /Users/pfiddes/Projects/apples/deps/prometheus_process_collector/c_src/prometheus_process_collector_backend.c:3:
/Users/pfiddes/Projects/apples/deps/prometheus_process_collector/c_src/prometheus_process_collector.h:9:10: fatal error: 'libutil.h' file not found
#include <libutil.h>
         ^~~~~~~~~~~
1 error generated.
make[1]: *** [/Users/pfiddes/Projects/apples/deps/prometheus_process_collector/c_src/prometheus_process_collector_backend.o] Error 1

Ignore me on alpine - just reviewed my output. They aren't warnings, just some output I thought might be useful.

[hawksight]

Just to confirm, ran on alpine container and it's working perfectly on linux (from master that is)

[deadtrickster]

Pushed one more time, no need for libutil.h now. Please check out master again.

[hawksight]

Tried again and different errors at least:

==> prometheus_process_collector
cc -O3 -std=c99 -arch x86_64 -finline-functions -Wall -Wmissing-prototypes -fPIC -I /usr/local/Cellar/erlang/20.1.2/lib/erlang/erts-9.1.2/include/ -I /usr/local/Cellar/erlang/20.1.2/lib/erlang/lib/erl_interface-3.10/include  -c -o /Users/pfiddes/Projects/apples/deps/prometheus_process_collector/c_src/prometheus_process_collector_backend.o /Users/pfiddes/Projects/apples/deps/prometheus_process_collector/c_src/prometheus_process_collector_backend.c
/Users/pfiddes/Projects/apples/deps/prometheus_process_collector/c_src/prometheus_process_collector_backend.c:19:1: warning: control reaches end of non-void function [-Wreturn-type]
}
^
/Users/pfiddes/Projects/apples/deps/prometheus_process_collector/c_src/prometheus_process_collector_backend.c:264:13: error: no member named 'ki_structsize' in 'struct kinfo_proc'
  if (kipp->ki_structsize != sizeof(*kipp))
      ~~~~  ^
/Users/pfiddes/Projects/apples/deps/prometheus_process_collector/c_src/prometheus_process_collector_backend.c:266:13: error: no member named 'ki_pid' in 'struct kinfo_proc'
  if (kipp->ki_pid != pid)
      ~~~~  ^
/Users/pfiddes/Projects/apples/deps/prometheus_process_collector/c_src/prometheus_process_collector_backend.c:286:6: warning: implicit declaration of function 'get_process_open_fd_totals' is invalid in C99 [-Wimplicit-function-declaration]
  if(get_process_open_fd_totals(pid, &pids_total)) {
     ^
/Users/pfiddes/Projects/apples/deps/prometheus_process_collector/c_src/prometheus_process_collector_backend.c:293:6: warning: implicit declaration of function 'get_process_limit' is invalid in C99 [-Wimplicit-function-declaration]
  if(get_process_limit(pid, RLIMIT_NOFILE, &rlimit)) {
     ^
/Users/pfiddes/Projects/apples/deps/prometheus_process_collector/c_src/prometheus_process_collector_backend.c:300:55: error: no member named 'ki_start' in 'struct kinfo_proc'
  prometheus_process_info->start_time_seconds = proc->ki_start.tv_sec;
                                                ~~~~  ^
/Users/pfiddes/Projects/apples/deps/prometheus_process_collector/c_src/prometheus_process_collector_backend.c:304:57: error: no member named 'ki_start' in 'struct kinfo_proc'
  prometheus_process_info->uptime_seconds = now - proc->ki_start.tv_sec;
                                                  ~~~~  ^
/Users/pfiddes/Projects/apples/deps/prometheus_process_collector/c_src/prometheus_process_collector_backend.c:306:50: error: no member named 'ki_numthreads' in 'struct kinfo_proc'
  prometheus_process_info->threads_total = proc->ki_numthreads;
                                           ~~~~  ^
/Users/pfiddes/Projects/apples/deps/prometheus_process_collector/c_src/prometheus_process_collector_backend.c:308:45: error: no member named 'ki_size' in 'struct kinfo_proc'
  prometheus_process_info->vm_bytes = proc->ki_size;
                                      ~~~~  ^
/Users/pfiddes/Projects/apples/deps/prometheus_process_collector/c_src/prometheus_process_collector_backend.c:310:45: error: no member named 'ki_rssize' in 'struct kinfo_proc'
  prometheus_process_info->rm_bytes = proc->ki_rssize * pagesize();
                                      ~~~~  ^
/Users/pfiddes/Projects/apples/deps/prometheus_process_collector/c_src/prometheus_process_collector_backend.c:312:32: error: no member named 'ki_rusage' in 'struct kinfo_proc'
  struct rusage rusage = proc->ki_rusage;
                         ~~~~  ^
3 warnings and 8 errors generated.
make[1]: *** [/Users/pfiddes/Projects/apples/deps/prometheus_process_collector/c_src/prometheus_process_collector_backend.o] Error 1
could not compile dependency :prometheus_process_collector, "mix compile" failed. You can recompile this dependency with "mix deps.compile prometheus_process_collector", update it with "mix deps.update prometheus_process_collector" or clean it with "mix deps.clean prometheus_process_collector"
==> apples
** (Mix) Could not compile with "make" (exit status: 2).
Depending on your OS, make sure to follow these instructions:

  * Mac OS X: You need to have gcc and make installed. Try running the
    commands "gcc --version" and / or "make --version". If these programs
    are not installed, you will be prompted to install them.

  * Linux: You need to have gcc and make installed. If you are using
    Ubuntu or any other Debian-based system, install the packages
    "build-essential". Also install "erlang-dev" package if not
    included in your Erlang/OTP version. If you're on Fedora, run
    "dnf group install 'Development Tools'".

Sorry I'm, not much help with C.

[deadtrickster]

Ok, this means more work has to be done for macos support. I'll post an update on this.

[hawksight]

Cool, thanks for the help still 👍 -> a few steps closer. I'm good with using it in container anyway as all ours end up on linux, Mac support is just helpful for our team's setup.

[deadtrickster]

@hawksight

Should work on master now. Please try!

[hawksight]

Hey, so I added this to mix.exs:

{:prometheus_process_collector, git: "git@github.com:deadtrickster/prometheus_process_collector.git", branch: "master", override: true},

But getting this:

==> prometheus_process_collector
Unchecked dependencies for environment prod:
* prometheus (Hex package)
  lock mismatch: the dependency is out of date. To fetch locked version run "mix deps.get"
could not compile dependency :prometheus_process_collector, "mix compile" failed. You can recompile this dependency with "mix deps.compile prometheus_process_collector", update it with "mix deps.update prometheus_process_collector" or clean it with "mix deps.clean prometheus_process_collector"
==> apples
** (Mix) Can't continue due to errors on dependencies

This is when trying to run mix deps.compile My full deps looks like:

  defp deps(_) do
    [
      {:phoenix, "~> 1.3.0"},
      {:phoenix_pubsub, "~> 1.0"},
      {:phoenix_ecto, "~> 3.2"},
      {:postgrex, ">= 0.0.0"},
      {:phoenix_html, "~> 2.10"},
      {:phoenix_live_reload, "~> 1.1.3", only: :dev},
      {:gettext, "~> 0.11"},
      {:cowboy, "~> 1.0"},
      {:prometheus, "~> 3.4.3"},
      {:prometheus_ex, "~> 1.0"},
      {:prometheus_plugs, "~> 1.0"},
      {:prometheus_ecto, "~> 1.0"},
      {:prometheus_phoenix, "~> 1.0"},
      {:prometheus_process_collector, git: "git@github.com:deadtrickster/prometheus_process_collector.git", branch: "master", override: true},
    ]
  end

[deadtrickster]

Hmm, why prod environment? anyway deleting mix.lock should help...

[hawksight]

yer tried that, and all the options in the output. Ok so changed it back to: {:prometheus_process_collector, "~> 1.2"} and everything worked normally (with hex anyway)

So the moment I change to git: {:prometheus_process_collector, git: "https://github.com/deadtrickster/prometheus_process_collector.git"}

mix deps.get outputs:

Dependencies have diverged:
* prometheus_process_collector (https://github.com/deadtrickster/prometheus_process_collector.git)
  the dependency prometheus_process_collector in mix.exs is overriding a child dependency:

  > In mix.exs:
    {:prometheus_process_collector, [env: :prod, git: "https://github.com/deadtrickster/prometheus_process_collector.git", manager: :mix]}

  > In deps/prometheus_plugs/mix.exs:
    {:prometheus_process_collector, "~> 1.1", [env: :prod, hex: "prometheus_process_collector", repo: "hexpm", optional: true]}

  Ensure they match or specify one of the above in your deps and set "override: true"
** (Mix) Can't continue due to errors on dependencies

So I override the dep: {:prometheus_process_collector, git: "https://github.com/deadtrickster/prometheus_process_collector.git", override: true} mix deps.get works but mix deps.compile or mix test breaks -> the unchecked dependency error. Tried cleaning, compiling, removing locks and still comes back to the same thing.

Does seem weird that is mentioning prod, but when I tried with 1.2.5 hex build it all worked fine.

[hawksight]

Kind of guessing here - but when i got into deps/prometheus_process_collector i pulled down the latest deps, then tried to compile the deps and got the following:

➜  prometheus_process_collector git:(caf3876) ✗ mix compile
** (Mix) The task "compile.elixir_make" could not be found. Did you mean "compile.elixir"?

just looking here I can see why elixir_make is commeneted out, it complains about make instalaltion if you bring it in.

Just wondering if the dep is failing to build because of the call to compile.elixir_make?

[deadtrickster]

I pushed 1.3.0 to hex.pm. Please try it. should resolve all mix issues (hex package doesn't have mix.exs at all and treated as plain rebar dependency)

[hawksight]

That worked so beautifully I had to screenshot it for you. And console

Thank you very much, this is awesome, really handy for me!

[deadtrickster]

Huh :-). Please let me know if you find any value is off. I'm still far from being MacOS X expert. And of course huge thank you (and others of course) for the help. Finally closing this!

[kenneth-duffel]

Getting the same issue on the silicon Mac books.

Could you help please:

The on_load function for module prometheus_process_collector returned: {:error, {:load_failed, 'Failed to load NIF library: \'dlopen(/_build/dev/lib/prometheus_process_collector/priv/prometheus_process_collector.so, 2): no suitable image found. Did find:\n\t/_build/dev/lib/prometheus_process_collector/priv/prometheus_process_collector.so: mach-o, but wrong architecture\n\t/_build/dev/lib/prometheus_process_collector/priv/prometheus_process_collector.so: stat() failed with errno=35\''}}

Request: GET /metrics (exit) an exception was raised: (UndefinedFunctionError) function :prometheus_process_collector.collect_mf/2 is undefined (module :prometheus_process_collector is not available) (prometheus_process_collector 1.6.0) :prometheus_process_collector.collect_mf(:default, #Function<1.18125299/1 in :prometheus_text_format.registry_collect_callback/3>) (prometheus 4.8.1) src/prometheus_collector.erl:156: :prometheus_collector.collect_mf/3 (prometheus 4.8.1) src/prometheus_registry.erl:86: :prometheus_registry."-collect/2-lc$^0/1-0-"/3 (prometheus 4.8.1) src/prometheus_registry.erl:87: :prometheus_registry.collect/2 (prometheus 4.8.1) src/formats/prometheus_text_format.erl:71: :prometheus_text_format.format/1 (duffel_web 0.0.1) lib/web/plugs/metrics_exporter_plug.ex:2: Duffel.Web.MetricsExporterPlug.scrape_data/1 (duffel_web 0.0.1) lib/web/plugs/metrics_exporter_plug.ex:2: Duffel.Web.MetricsExporterPlug.call/2 (duffel_web 0.0.1) lib/web/endpoints/metrics_endpoint.ex:1: Duffel.Web.MetricsEndpoint.plug_builder_call/2 (duffel_web 0.0.1) lib/web/endpoints/metrics_endpoint.ex:1: Duffel.Web.MetricsEndpoint.call/2 (phoenix 1.6.2) lib/phoenix/endpoint/cowboy2_handler.ex:43: Phoenix.Endpoint.Cowboy2Handler.init/4 (cowboy 2.9.0) /deps/cowboy/src/cowboy_handler.erl:37: :cowboy_handler.execute/2 (cowboy 2.9.0) /deps/cowboy/src/cowboy_stream_h.erl:306: :cowboy_stream_h.execute/3 (cowboy 2.9.0) /deps/cowboy/src/cowboy_stream_h.erl:295: :cowboy_stream_h.request_process/3 (stdlib 3.16.1) proc_lib.erl:226: :proc_lib.init_p_do_apply/3