Certifi module not found

[micke]

Getting this error when trying to use nostrum. I'm pretty new to elixir and the beam so not 100% sure if this is the right place to file this issue. From what i can gather it's the same issue as in #524. I've tried to override the certifi version in my mix.exs by using: {:certifi, "~> 2.4 and >= 2.4.2", override: true}.

I currently fix it by running mix deps.compile certifi. But it feels like a really dirty solution.

Elixir v1.7.3 Erlang 21

Erlang/OTP 21 [erts-10.0.6] [source] [64-bit] [smp:8:8] [ds:8:8:10] [async-threads:1] [hipe]

[error] GenServer Ratelimiter terminating
** (UndefinedFunctionError) function :certifi.cacerts/0 is undefined (module :certifi is not available)
    (certifi) :certifi.cacerts()
    (hackney) /Users/lisinge/code/lisinge/munkey/deps/hackney/src/hackney_connect.erl:296: :hackney_connect.ssl_opts_1/2
    (hackney) /Users/lisinge/code/lisinge/munkey/deps/hackney/src/hackney_connect.erl:243: :hackney_connect.do_connect/5
    (hackney) /Users/lisinge/code/lisinge/munkey/deps/hackney/src/hackney_connect.erl:37: :hackney_connect.connect/5
    (hackney) /Users/lisinge/code/lisinge/munkey/deps/hackney/src/hackney.erl:316: :hackney.request/5
    (httpoison) lib/httpoison/base.ex:439: HTTPoison.Base.request/9
    (nostrum) lib/nostrum/api/ratelimiter.ex:72: Nostrum.Api.Ratelimiter.do_request/1
    (nostrum) lib/nostrum/api/ratelimiter.ex:56: Nostrum.Api.Ratelimiter.handle_call/3
    (stdlib) gen_server.erl:661: :gen_server.try_handle_call/4
    (stdlib) gen_server.erl:690: :gen_server.handle_msg/6
    (stdlib) proc_lib.erl:249: :proc_lib.init_p_do_apply/3
Last message (from #PID<0.286.0>): {:queue, %{body: "", headers: [{"content-type", "application/json"}], method: :get, options: [], route: "/gateway/bot"}, nil}
State: []
Client #PID<0.286.0> is alive

    (stdlib) gen.erl:169: :gen.do_call/4
    (elixir) lib/gen_server.ex:921: GenServer.call/3
    (nostrum) lib/nostrum/util.ex:154: Nostrum.Util.get_new_gateway_url/0
    (nostrum) lib/nostrum/shard/supervisor.ex:13: Nostrum.Shard.Supervisor.start_link/1
    (stdlib) supervisor.erl:379: :supervisor.do_start_child_i/3
    (stdlib) supervisor.erl:365: :supervisor.do_start_child/2
    (stdlib) supervisor.erl:349: anonymous fn/3 in :supervisor.start_children/2
    (stdlib) supervisor.erl:1157: :supervisor.children_map/4
    (stdlib) supervisor.erl:315: :supervisor.init_children/2
    (stdlib) gen_server.erl:374: :gen_server.init_it/2
    (stdlib) gen_server.erl:342: :gen_server.init_it/6
    (stdlib) proc_lib.erl:249: :proc_lib.init_p_do_apply/3
** (Mix) Could not start application nostrum: Nostrum.Application.start(:normal, []) returned an error: shutdown: failed to start child: Nostrum.Shard.Supervisor
    ** (EXIT) exited in: GenServer.call(Ratelimiter, {:queue, %{body: "", headers: [{"content-type", "application/json"}], method: :get, options: [], route: "/gateway/bot"}, nil}, :infinity)
        ** (EXIT) an exception was raised:
            ** (UndefinedFunctionError) function :certifi.cacerts/0 is undefined (module :certifi is not available)
                (certifi) :certifi.cacerts()
                (hackney) /Users/lisinge/code/lisinge/munkey/deps/hackney/src/hackney_connect.erl:296: :hackney_connect.ssl_opts_1/2
                (hackney) /Users/lisinge/code/lisinge/munkey/deps/hackney/src/hackney_connect.erl:243: :hackney_connect.do_connect/5
                (hackney) /Users/lisinge/code/lisinge/munkey/deps/hackney/src/hackney_connect.erl:37: :hackney_connect.connect/5
                (hackney) /Users/lisinge/code/lisinge/munkey/deps/hackney/src/hackney.erl:316: :hackney.request/5
                (httpoison) lib/httpoison/base.ex:439: HTTPoison.Base.request/9
                (nostrum) lib/nostrum/api/ratelimiter.ex:72: Nostrum.Api.Ratelimiter.do_request/1
                (nostrum) lib/nostrum/api/ratelimiter.ex:56: Nostrum.Api.Ratelimiter.handle_call/3
                (stdlib) gen_server.erl:661: :gen_server.try_handle_call/4
                (stdlib) gen_server.erl:690: :gen_server.handle_msg/6
                (stdlib) proc_lib.erl:249: :proc_lib.init_p_do_apply/3

[benoitc]

do you reproduce it with latest master?

[benoitc]

bump

[micke]

Hi @benoitc thanks for getting back so quickly. I just tried pointing hackney to the latest master and still getting the same error.

~/code/lisinge/munkey $ iex -S mix
Erlang/OTP 21 [erts-10.0.6] [source] [64-bit] [smp:8:8] [ds:8:8:10] [async-threads:1] [hipe]

[error] GenServer Ratelimiter terminating
** (UndefinedFunctionError) function :certifi.cacerts/0 is undefined (module :certifi is not available)
    (certifi) :certifi.cacerts()
    (hackney) /Users/lisinge/code/lisinge/munkey/deps/hackney/src/hackney_connect.erl:321: :hackney_connect.ssl_opts_1/2
    (hackney) /Users/lisinge/code/lisinge/munkey/deps/hackney/src/hackney_connect.erl:268: :hackney_connect.do_connect/5
    (hackney) /Users/lisinge/code/lisinge/munkey/deps/hackney/src/hackney_connect.erl:39: :hackney_connect.connect/5
    (hackney) /Users/lisinge/code/lisinge/munkey/deps/hackney/src/hackney.erl:328: :hackney.request/5
    (httpoison) lib/httpoison/base.ex:439: HTTPoison.Base.request/9
    (nostrum) lib/nostrum/api/ratelimiter.ex:72: Nostrum.Api.Ratelimiter.do_request/1
    (nostrum) lib/nostrum/api/ratelimiter.ex:56: Nostrum.Api.Ratelimiter.handle_call/3
    (stdlib) gen_server.erl:661: :gen_server.try_handle_call/4
    (stdlib) gen_server.erl:690: :gen_server.handle_msg/6
    (stdlib) proc_lib.erl:249: :proc_lib.init_p_do_apply/3
Last message (from #PID<0.288.0>): {:queue, %{body: "", headers: [{"content-type", "application/json"}], method: :get, options: [], route: "/gateway/bot"}, nil}
State: []
Client #PID<0.288.0> is alive

    (stdlib) gen.erl:169: :gen.do_call/4
    (elixir) lib/gen_server.ex:921: GenServer.call/3
    (nostrum) lib/nostrum/util.ex:154: Nostrum.Util.get_new_gateway_url/0
    (nostrum) lib/nostrum/shard/supervisor.ex:13: Nostrum.Shard.Supervisor.start_link/1
    (stdlib) supervisor.erl:379: :supervisor.do_start_child_i/3
    (stdlib) supervisor.erl:365: :supervisor.do_start_child/2
    (stdlib) supervisor.erl:349: anonymous fn/3 in :supervisor.start_children/2
    (stdlib) supervisor.erl:1157: :supervisor.children_map/4
    (stdlib) supervisor.erl:315: :supervisor.init_children/2
    (stdlib) gen_server.erl:374: :gen_server.init_it/2
    (stdlib) gen_server.erl:342: :gen_server.init_it/6
    (stdlib) proc_lib.erl:249: :proc_lib.init_p_do_apply/3
** (Mix) Could not start application nostrum: Nostrum.Application.start(:normal, []) returned an error: shutdown: failed to start child: Nostrum.Shard.Supervisor
    ** (EXIT) exited in: GenServer.call(Ratelimiter, {:queue, %{body: "", headers: [{"content-type", "application/json"}], method: :get, options: [], route: "/gateway/bot"}, nil}, :infinity)
        ** (EXIT) an exception was raised:
            ** (UndefinedFunctionError) function :certifi.cacerts/0 is undefined (module :certifi is not available)
                (certifi) :certifi.cacerts()
                (hackney) /Users/lisinge/code/lisinge/munkey/deps/hackney/src/hackney_connect.erl:321: :hackney_connect.ssl_opts_1/2
                (hackney) /Users/lisinge/code/lisinge/munkey/deps/hackney/src/hackney_connect.erl:268: :hackney_connect.do_connect/5
                (hackney) /Users/lisinge/code/lisinge/munkey/deps/hackney/src/hackney_connect.erl:39: :hackney_connect.connect/5
                (hackney) /Users/lisinge/code/lisinge/munkey/deps/hackney/src/hackney.erl:328: :hackney.request/5
                (httpoison) lib/httpoison/base.ex:439: HTTPoison.Base.request/9
                (nostrum) lib/nostrum/api/ratelimiter.ex:72: Nostrum.Api.Ratelimiter.do_request/1
                (nostrum) lib/nostrum/api/ratelimiter.ex:56: Nostrum.Api.Ratelimiter.handle_call/3
                (stdlib) gen_server.erl:661: :gen_server.try_handle_call/4
                (stdlib) gen_server.erl:690: :gen_server.handle_msg/6
                (stdlib) proc_lib.erl:249: :proc_lib.init_p_do_apply/3

[micke]

It's cumbersome because it's not happening consistently. But running tests or the iex enough times ends up with this error

EDIT; let me know if i should open this issue in the certifi repo instead or if i can help in any other way to debug this.

[mhanberg]

I am seeing the same error after upgrading to elixir v1.7.3 and erlang v21

mix.lock

%{
  "bunt": {:hex, :bunt, "0.2.0", "951c6e801e8b1d2cbe58ebbd3e616a869061ddadcc4863d0a2182541acae9a38", [:mix], [], "hexpm"},
  "certifi": {:hex, :certifi, "2.3.1", "d0f424232390bf47d82da8478022301c561cf6445b5b5fb6a84d49a9e76d2639", [:rebar3], [{:parse_trans, "3.2.0", [hex: :parse_trans, repo: "hexpm", optional: false]}], "hexpm"},
  "combine": {:hex, :combine, "0.10.0", "eff8224eeb56498a2af13011d142c5e7997a80c8f5b97c499f84c841032e429f", [:mix], [], "hexpm"},
  "connection": {:hex, :connection, "1.0.4", "a1cae72211f0eef17705aaededacac3eb30e6625b04a6117c1b2db6ace7d5976", [:mix], [], "hexpm"},
  "cowboy": {:hex, :cowboy, "1.1.2", "61ac29ea970389a88eca5a65601460162d370a70018afe6f949a29dca91f3bb0", [:rebar3], [{:cowlib, "~> 1.0.2", [hex: :cowlib, repo: "hexpm", optional: false]}, {:ranch, "~> 1.3.2", [hex: :ranch, repo: "hexpm", optional: false]}], "hexpm"},
  "cowlib": {:hex, :cowlib, "1.0.2", "9d769a1d062c9c3ac753096f868ca121e2730b9a377de23dec0f7e08b1df84ee", [:make], [], "hexpm"},
  "credo": {:hex, :credo, "0.10.0", "66234a95effaf9067edb19fc5d0cd5c6b461ad841baac42467afed96c78e5e9e", [:mix], [{:bunt, "~> 0.2.0", [hex: :bunt, repo: "hexpm", optional: false]}, {:jason, "~> 1.0", [hex: :jason, repo: "hexpm", optional: false]}], "hexpm"},
  "db_connection": {:hex, :db_connection, "1.1.3", "89b30ca1ef0a3b469b1c779579590688561d586694a3ce8792985d4d7e575a61", [:mix], [{:connection, "~> 1.0.2", [hex: :connection, repo: "hexpm", optional: false]}, {:poolboy, "~> 1.5", [hex: :poolboy, repo: "hexpm", optional: true]}, {:sbroker, "~> 1.0", [hex: :sbroker, repo: "hexpm", optional: true]}], "hexpm"},
  "decimal": {:hex, :decimal, "1.5.0", "b0433a36d0e2430e3d50291b1c65f53c37d56f83665b43d79963684865beab68", [:mix], [], "hexpm"},
  "dotenv": {:hex, :dotenv, "2.0.0", "87e77a94e54c20eca49989876395e26878b109f016f9fd9c42b65bb9c59bca3f", [:mix], [], "hexpm"},
  "ecto": {:hex, :ecto, "2.2.10", "e7366dc82f48f8dd78fcbf3ab50985ceeb11cb3dc93435147c6e13f2cda0992e", [:mix], [{:db_connection, "~> 1.1", [hex: :db_connection, repo: "hexpm", optional: true]}, {:decimal, "~> 1.2", [hex: :decimal, repo: "hexpm", optional: false]}, {:mariaex, "~> 0.8.0", [hex: :mariaex, repo: "hexpm", optional: true]}, {:poison, "~> 2.2 or ~> 3.0", [hex: :poison, repo: "hexpm", optional: true]}, {:poolboy, "~> 1.5", [hex: :poolboy, repo: "hexpm", optional: false]}, {:postgrex, "~> 0.13.0", [hex: :postgrex, repo: "hexpm", optional: true]}, {:sbroker, "~> 1.0", [hex: :sbroker, repo: "hexpm", optional: true]}], "hexpm"},
  "file_system": {:hex, :file_system, "0.2.6", "fd4dc3af89b9ab1dc8ccbcc214a0e60c41f34be251d9307920748a14bf41f1d3", [:mix], [], "hexpm"},
  "gettext": {:hex, :gettext, "0.16.0", "4a7e90408cef5f1bf57c5a39e2db8c372a906031cc9b1466e963101cb927dafc", [:mix], [], "hexpm"},
  "hackney": {:hex, :hackney, "1.13.0", "24edc8cd2b28e1c652593833862435c80661834f6c9344e84b6a2255e7aeef03", [:rebar3], [{:certifi, "2.3.1", [hex: :certifi, repo: "hexpm", optional: false]}, {:idna, "5.1.2", [hex: :idna, repo: "hexpm", optional: false]}, {:metrics, "1.0.1", [hex: :metrics, repo: "hexpm", optional: false]}, {:mimerl, "1.0.2", [hex: :mimerl, repo: "hexpm", optional: false]}, {:ssl_verify_fun, "1.1.1", [hex: :ssl_verify_fun, repo: "hexpm", optional: false]}], "hexpm"},
  "httpoison": {:hex, :httpoison, "1.3.0", "edfd6ca53b57377a7a8db226bc42a2bddfeb39951c61ddfe15ec930225e6e46b", [:mix], [{:hackney, "~> 1.8", [hex: :hackney, repo: "hexpm", optional: false]}], "hexpm"},
  "idna": {:hex, :idna, "5.1.2", "e21cb58a09f0228a9e0b95eaa1217f1bcfc31a1aaa6e1fdf2f53a33f7dbd9494", [:rebar3], [{:unicode_util_compat, "0.3.1", [hex: :unicode_util_compat, repo: "hexpm", optional: false]}], "hexpm"},
  "jason": {:hex, :jason, "1.1.1", "d3ccb840dfb06f2f90a6d335b536dd074db748b3e7f5b11ab61d239506585eb2", [:mix], [{:decimal, "~> 1.0", [hex: :decimal, repo: "hexpm", optional: true]}], "hexpm"},
  "metrics": {:hex, :metrics, "1.0.1", "25f094dea2cda98213cecc3aeff09e940299d950904393b2a29d191c346a8486", [:rebar3], [], "hexpm"},
  "mime": {:hex, :mime, "1.3.0", "5e8d45a39e95c650900d03f897fbf99ae04f60ab1daa4a34c7a20a5151b7a5fe", [:mix], [], "hexpm"},
  "mimerl": {:hex, :mimerl, "1.0.2", "993f9b0e084083405ed8252b99460c4f0563e41729ab42d9074fd5e52439be88", [:rebar3], [], "hexpm"},
  "mox": {:hex, :mox, "0.3.2", "3b9b8364fd4f28628139de701d97c636b27a8f925f57a8d5a1b85fbd620dad3a", [:mix], [], "hexpm"},
  "parse_trans": {:hex, :parse_trans, "3.2.0", "2adfa4daf80c14dc36f522cf190eb5c4ee3e28008fc6394397c16f62a26258c2", [:rebar3], [], "hexpm"},
  "phoenix": {:hex, :phoenix, "1.3.4", "aaa1b55e5523083a877bcbe9886d9ee180bf2c8754905323493c2ac325903dc5", [:mix], [{:cowboy, "~> 1.0", [hex: :cowboy, repo: "hexpm", optional: true]}, {:phoenix_pubsub, "~> 1.0", [hex: :phoenix_pubsub, repo: "hexpm", optional: false]}, {:plug, "~> 1.3.3 or ~> 1.4", [hex: :plug, repo: "hexpm", optional: false]}, {:poison, "~> 2.2 or ~> 3.0", [hex: :poison, repo: "hexpm", optional: false]}], "hexpm"},
  "phoenix_ecto": {:hex, :phoenix_ecto, "3.4.0", "91cd39427006fe4b5588d69f0941b9c3d3d8f5e6477c563a08379de7de2b0c58", [:mix], [{:ecto, "~> 2.1", [hex: :ecto, repo: "hexpm", optional: false]}, {:phoenix_html, "~> 2.9", [hex: :phoenix_html, repo: "hexpm", optional: true]}, {:plug, "~> 1.0", [hex: :plug, repo: "hexpm", optional: false]}], "hexpm"},
  "phoenix_html": {:hex, :phoenix_html, "2.12.0", "1fb3c2e48b4b66d75564d8d63df6d53655469216d6b553e7e14ced2b46f97622", [:mix], [{:plug, "~> 1.5", [hex: :plug, repo: "hexpm", optional: false]}], "hexpm"},
  "phoenix_live_reload": {:hex, :phoenix_live_reload, "1.1.5", "8d4c9b1ef9ca82deee6deb5a038d6d8d7b34b9bb909d99784a49332e0d15b3dc", [:mix], [{:file_system, "~> 0.2.1 or ~> 0.3", [hex: :file_system, repo: "hexpm", optional: false]}, {:phoenix, "~> 1.0 or ~> 1.2 or ~> 1.3", [hex: :phoenix, repo: "hexpm", optional: false]}], "hexpm"},
  "phoenix_pubsub": {:hex, :phoenix_pubsub, "1.1.0", "d55e25ff1ff8ea2f9964638366dfd6e361c52dedfd50019353598d11d4441d14", [:mix], [], "hexpm"},
  "plug": {:hex, :plug, "1.6.2", "e06a7bd2bb6de5145da0dd950070110dce88045351224bd98e84edfdaaf5ffee", [:mix], [{:cowboy, "~> 1.0.1 or ~> 1.1 or ~> 2.4", [hex: :cowboy, repo: "hexpm", optional: true]}, {:mime, "~> 1.0", [hex: :mime, repo: "hexpm", optional: false]}], "hexpm"},
  "poison": {:hex, :poison, "3.1.0", "d9eb636610e096f86f25d9a46f35a9facac35609a7591b3be3326e99a0484665", [:mix], [], "hexpm"},
  "poolboy": {:hex, :poolboy, "1.5.1", "6b46163901cfd0a1b43d692657ed9d7e599853b3b21b95ae5ae0a777cf9b6ca8", [:rebar], [], "hexpm"},
  "postgrex": {:hex, :postgrex, "0.13.5", "3d931aba29363e1443da167a4b12f06dcd171103c424de15e5f3fc2ba3e6d9c5", [:mix], [{:connection, "~> 1.0", [hex: :connection, repo: "hexpm", optional: false]}, {:db_connection, "~> 1.1", [hex: :db_connection, repo: "hexpm", optional: false]}, {:decimal, "~> 1.0", [hex: :decimal, repo: "hexpm", optional: false]}], "hexpm"},
  "ranch": {:hex, :ranch, "1.3.2", "e4965a144dc9fbe70e5c077c65e73c57165416a901bd02ea899cfd95aa890986", [:rebar3], [], "hexpm"},
  "ssl_verify_fun": {:hex, :ssl_verify_fun, "1.1.1", "28a4d65b7f59893bc2c7de786dec1e1555bd742d336043fe644ae956c3497fbe", [:make, :rebar], [], "hexpm"},
  "sweet_xml": {:git, "git@github.com:mhanberg/sweet_xml.git", "cd217a514105f18d95b658045759e61bae6d3041", []},
  "timex": {:hex, :timex, "3.3.0", "e0695aa0ddb37d460d93a2db34d332c2c95a40c27edf22fbfea22eb8910a9c8d", [:mix], [{:combine, "~> 0.10", [hex: :combine, repo: "hexpm", optional: false]}, {:gettext, "~> 0.10", [hex: :gettext, repo: "hexpm", optional: false]}, {:tzdata, "~> 0.1.8 or ~> 0.5", [hex: :tzdata, repo: "hexpm", optional: false]}], "hexpm"},
  "tzdata": {:hex, :tzdata, "0.5.19", "7962a3997bf06303b7d1772988ede22260f3dae1bf897408ebdac2b4435f4e6a", [:mix], [{:hackney, "~> 1.0", [hex: :hackney, repo: "hexpm", optional: false]}], "hexpm"},
  "unicode_util_compat": {:hex, :unicode_util_compat, "0.3.1", "a1f612a7b512638634a603c8f401892afbf99b8ce93a45041f8aaca99cadb85e", [:rebar3], [], "hexpm"},
  "wallaby": {:hex, :wallaby, "0.20.0", "cc6663555ff7b05afbebb2a8b461d18a5b321658b9017f7bc77d494b7063266a", [:mix], [{:httpoison, "~> 0.12", [hex: :httpoison, repo: "hexpm", optional: false]}, {:poison, ">= 1.4.0", [hex: :poison, repo: "hexpm", optional: false]}, {:poolboy, "~> 1.5", [hex: :poolboy, repo: "hexpm", optional: false]}], "hexpm"},
}

** (Mix) Could not start application planet: Planet.Application.start(:normal, []) returned an error: shutdown: failed to start child: Planet.Core.ServerFarmSupervisor
    ** (EXIT) shutdown: failed to start child: Planet.Core.FeedStore
        ** (EXIT) an exception was raised:
            ** (MatchError) no match of right hand side value: {:error, {:undef, [{:certifi, :cacerts, [], []}, {:hackney_connect, :ssl_opts_1, 2, [file: '/Users/mitchell/Development/planet/deps/hackney/src/hackney_connect.erl', line: 296]}, {:hackney_connect, :do_connect, 5, [file: '/Users/mitchell/Development/planet/deps/hackney/src/hackney_connect.erl', line: 243]}, {:hackney, :send_request, 2, [file: '/Users/mitchell/Development/planet/deps/hackney/src/hackney.erl', line: 355]}, {:hackney, :redirect, 2, [file: '/Users/mitchell/Development/planet/deps/hackney/src/hackney.erl', line: 811]}, {:hackney, :maybe_redirect1, 3, [file: '/Users/mitchell/Development/planet/deps/hackney/src/hackney.erl', line: 743]}, {:hackney, :send_request, 2, [file: '/Users/mitchell/Development/planet/deps/hackney/src/hackney.erl', line: 363]}, {HTTPoison.Base, :request, 9, [file: 'lib/httpoison/base.ex', line: 632]}, {Planet.Core.FeedFetcher, :get, 1, [file: 'lib/planet/core/feed_fetcher.ex', line: 11]}, {Planet.Core.FeedServer, :build_feed, 1, [file: 'lib/planet/core/feed_server.ex', line: 75]}, {Planet.Core.FeedServer, :init, 1, [file: 'lib/planet/core/feed_server.ex', line: 36]}, {:gen_server, :init_it, 2, [file: 'gen_server.erl', line: 374]}, {:gen_server, :init_it, 6, [file: 'gen_server.erl', line: 342]}, {:proc_lib, :init_p_do_apply, 3, [file: 'proc_lib.erl', line: 249]}]}}
                (planet) lib/planet/core/feed_store.ex:44: anonymous fn/2 in Planet.Core.FeedStore.init/1
                (elixir) lib/enum.ex:1925: Enum."-reduce/3-lists^foldl/2-0-"/3
                (planet) lib/planet/core/feed_store.ex:43: Planet.Core.FeedStore.init/1
                (stdlib) gen_server.erl:374: :gen_server.init_it/2
                (stdlib) gen_server.erl:342: :gen_server.init_it/6
                (stdlib) proc_lib.erl:249: :proc_lib.init_p_do_apply/3

Mine is throwing inside of a call to DynamicSupervisor.start_child, so I believe it rescues the error internally and returns it, which is why my error is a MatchError.

I am using the elixir library HTTPoison, which internally uses Hackney.

[benoitc]

did you try with the latest version of hackney?

[mhanberg]

Whoops, sorry for being that person 😅, I ran mix deps.update hackney (which i didn't think to since its a dependency of a dependency) and now it looks to be working.

Upgraded:
  certifi 2.3.1 => 2.4.2
  hackney 1.13.0 => 1.14.0
  idna 5.1.2 => 6.0.0
  parse_trans 3.2.0 => 3.3.0
  ssl_verify_fun 1.1.1 => 1.1.4
  unicode_util_compat 0.3.1 => 0.4.1

Sorry for contributing to the noise!

[aaronrenner]

I am also seeing this issue. It seems to work the first time or two after I compile my app, but then it stops working and I get the following error:

10:05:31.864 [error] GenServer Honeybadger.Client terminating
** (UndefinedFunctionError) function :certifi.cacerts/0 is undefined (module :certifi is not available)
    (certifi) :certifi.cacerts()
    (hackney) /Users/aaron/projects/honeydew_honeybadger/deps/hackney/src/hackney_connect.erl:321: :hackney_connect.ssl_opts_1/2
    (hackney) /Users/aaron/projects/honeydew_honeybadger/deps/hackney/src/hackney_connect.erl:268: :hackney_connect.do_connect/5
    (hackney) /Users/aaron/projects/honeydew_honeybadger/deps/hackney/src/hackney_connect.erl:39: :hackney_connect.connect/5
    (hackney) /Users/aaron/projects/honeydew_honeybadger/deps/hackney/src/hackney.erl:329: :hackney.request/5
    (honeybadger) lib/honeybadger/client.ex:145: Honeybadger.Client.post_notice/4
    (honeybadger) lib/honeybadger/client.ex:122: Honeybadger.Client.handle_cast/2
    (stdlib) gen_server.erl:637: :gen_server.try_dispatch/4

I know it says honeybadger, but it's using hackney under the hood. Here's the versions I'm running

Elixir: 1.7.3 OTP: 21.0.5 Hackney: master (commit daf085b) Certifi: master (commit 4ad4051 (2.4.2))

The way I fix this is rm -rf _build and then run my app again. Then it will work for a few boots of my app and then stop working. Not sure what's going on.

[mhanberg]

I am actually running into this problem again, seeing the same behavior as @aaronrenner, with needing to destroy my _build directory everytime or run mix deps.clean certifi; mix deps.get on every build.

[micke]

I'm no erlang expert but I'm pretty sure we are all experiencing the same error 😅. The bug is very inconsistent. Sometimes i don't get it for a whole evening and the next day i have to recompile cacerts every time i restart iex.

[rupurt]

Also seeing the problem on OTP 21/Elixir 1.7.3. It happens whenever I make a change to my source and then try to run iex. I have been working around it by recompiling the dependencies before running iex.

mix deps.compile && iex -S mix

[craigp]

This is happening to me as well, as of this morning. Upgraded hackney to 1.14.3 from 1.14.0 and it still happens. mix clean and mix clean.deps hackney had no effect, but as with @rupurt using mix deps.compile first seems to fix it, for a while at least

[benoitc]

i still have to reproduce it. Do you have any simple example that i could test ? That would help a lot :)

[bugant]

Here is a simple repro using hackney 1.14.3 and certifi 2.4.2

Erlang/OTP 21 [erts-10.0.5] [source] [64-bit] [smp:4:4] [ds:4:4:10] [async-threads:1] [hipe]

Interactive Elixir (1.7.3) - press Ctrl+C to exit (type h() ENTER for help)
iex(1)> HTTPoison.get("https://google.com")                                                                                                                                                                                    
** (UndefinedFunctionError) function :certifi.cacerts/0 is undefined (module :certifi is not available)
    (certifi) :certifi.cacerts()
    (hackney) /private/tmp/foo_app/deps/hackney/src/hackney_connect.erl:326: :hackney_connect.ssl_opts_1/2
    (hackney) /private/tmp/foo_app/deps/hackney/src/hackney_connect.erl:273: :hackney_connect.do_connect/5
    (hackney) /private/tmp/foo_app/deps/hackney/src/hackney_connect.erl:45: :hackney_connect.connect/5
    (hackney) /private/tmp/foo_app/deps/hackney/src/hackney.erl:329: :hackney.request/5
    (httpoison) lib/httpoison/base.ex:633: HTTPoison.Base.request/9```

[benoitc]

@bugant can you share your mix conf?

[bugant]

Here is my mix.lock

%{
  "certifi": {:hex, :certifi, "2.4.2", "75424ff0f3baaccfd34b1214184b6ef616d89e420b258bb0a5ea7d7bc628f7f0", [:rebar3], [{:parse_trans, "~>3.3", [hex: :parse_trans, repo: "hexpm", optional: false]}], "hexpm"},
  "hackney": {:hex, :hackney, "1.14.3", "b5f6f5dcc4f1fba340762738759209e21914516df6be440d85772542d4a5e412", [:rebar3], [{:certifi, "2.4.2", [hex: :certifi, repo: "hexpm", optional: false]}, {:idna, "6.0.0", [hex: :idna, repo: "hexpm", optional: false]}, {:metrics, "1.0.1", [hex: :metrics, repo: "hexpm", optional: false]}, {:mimerl, "1.0.2", [hex: :mimerl, repo: "hexpm", optional: false]}, {:ssl_verify_fun, "1.1.4", [hex: :ssl_verify_fun, repo: "hexpm", optional: false]}], "hexpm"},
  "httpoison": {:hex, :httpoison, "1.3.1", "7ac607311f5f706b44e8b3fab736d0737f2f62a31910ccd9afe7227b43edb7f0", [:mix], [{:hackney, "~> 1.8", [hex: :hackney, repo: "hexpm", optional: false]}], "hexpm"},
  "idna": {:hex, :idna, "6.0.0", "689c46cbcdf3524c44d5f3dde8001f364cd7608a99556d8fbd8239a5798d4c10", [:rebar3], [{:unicode_util_compat, "0.4.1", [hex: :unicode_util_compat, repo: "hexpm", optional: false]}], "hexpm"},
  "jason": {:hex, :jason, "1.1.1", "d3ccb840dfb06f2f90a6d335b536dd074db748b3e7f5b11ab61d239506585eb2", [:mix], [{:decimal, "~> 1.0", [hex: :decimal, repo: "hexpm", optional: true]}], "hexpm"},
  "metrics": {:hex, :metrics, "1.0.1", "25f094dea2cda98213cecc3aeff09e940299d950904393b2a29d191c346a8486", [:rebar3], [], "hexpm"},
  "mimerl": {:hex, :mimerl, "1.0.2", "993f9b0e084083405ed8252b99460c4f0563e41729ab42d9074fd5e52439be88", [:rebar3], [], "hexpm"},
  "parse_trans": {:hex, :parse_trans, "3.3.0", "09765507a3c7590a784615cfd421d101aec25098d50b89d7aa1d66646bc571c1", [:rebar3], [], "hexpm"},
  "ssl_verify_fun": {:hex, :ssl_verify_fun, "1.1.4", "f0eafff810d2041e93f915ef59899c923f4568f4585904d010387ed74988e77b", [:make, :mix, :rebar3], [], "hexpm"},
  "unicode_util_compat": {:hex, :unicode_util_compat, "0.4.1", "d869e4c68901dd9531385bb0c8c40444ebf624e60b6962d95952775cac5e90cd", [:rebar3], [], "hexpm"},
}

[benoitc]

Thanks but I was looking for a simple elixir project to test. I will try this afternoon anyway. I am actually wondering if something changed in the way elixir is loading projects but I will check.

[micke]

Thanks but I was looking for a simple elixir project to test. I will try this afternoon anyway. I am actually wondering if something changed in the way elixir is loading projects but I will check.

@benoitc I can invite you to my small elixir app that exhibits the problem if that would help.

[benoitc]

@micke if it's possible i can try over it also. more example are better than one :)

[micke]

@benoitc just added you. The error happens quite regularly when just running iex -S mix

[benoitc]

@micke just saw it. I will update the tixket later in the afternoon. Thanks!

[glanotte]

Just in the event this is helpful, I see the error when using elixir 1.7.x but not on 1.6.x

[rupurt]

@benoitc you can try it out on my project

• Run EXAMPLES=true iex -S mix
• Close the session
• Add a new line somewhere e.g. IO.puts "Hello World"
• Try to run the iex session again and you should see the certifi error

[rupurt]

@benoitc I did a little digging and it looks like the problem is that the certifi.beam file gets deleted at some point. If you run mix deps.compile you should see something like this in your dependencies.

_build/dev/lib/certifi/ebin
$ ls -lH _build/dev/lib/certifi/ebin
total 792
-rw-r--r--  1 alex  staff     540 Oct  2 21:25 certifi.app
-rw-r--r--  1 alex  staff  401144 Oct 12 13:47 certifi.beam

When I make a change it gets deleted. The timing seems to be a little sporadic but I can replicate the deletion every time by making a change, then starting a new iex session. I monitor the files with watch -n 0.1 ls -lH _build/dev/lib/certifi/ebin and after a few seconds it gets deleted.

$ ls -lH _build/dev/lib/certifi/ebin
total 8
-rw-r--r--  1 alex  staff  540 Oct  2 21:25 certifi.app

I'm not really sure where I should look next as erlang-certifi seems pretty simple and doesn't appear to delete the .beam from what I can see.

[benoitc]

it's not always reproducible. i am wondering if it's not an issue with the parse_transform, since parse_trans is not part of the app.src. I will bump certifi and retest. Expect a new release tomorrow anyway.

[rupurt]

@benoitc I tested out the new changes in https://github.com/certifi/erlang-certifi/commit/00e3759068a4c7d075340f45a67aef5e4ed0bebe but unfortunately it didn't fix the problem :{

[benoitc]

yes i’m still trying to figure what’s happening. i will have it fixed this we i guess.

On Friday, October 19, 2018, Alex Kwiatkowski notifications@github.com wrote:

@benoitc https://github.com/benoitc I tested out the new changes in certifi/erlang-certifi@00e3759 https://github.com/certifi/erlang-certifi/commit/00e3759068a4c7d075340f45a67aef5e4ed0bebe but unfortunately it didn't fix the problem :{

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/benoitc/hackney/issues/528#issuecomment-431407768, or mute the thread https://github.com/notifications/unsubscribe-auth/AAA4ol5pVcy46lpVfbTzOTmry-o3dZc1ks5umfN4gaJpZM4WkAaf .

-- Sent from my Mobile

[rupurt]

@benoitc no worries. Thanks a million for looking into it. Is there any other way I can help?

[sublimecoder]

Any further developments on this? I'm getting this error on several apps not sure how to go about fixing it.

[benoitc]

@sublimecoder i will pend some cycle today and provide a version that will eventually fix it. I don't think it's related to certifi itself though but may be due to the current double support of rebar2/rebar3 that make use use a script file.

[benoitc]

hrm I couldn't reproduce it. Are you running mix with rebar3 installed?

[dalhorinek]

I'm having same issue actually.

elixir -v
Erlang/OTP 21 [erts-10.0.5] [source] [64-bit] [smp:4:4] [ds:4:4:10] [async-threads:1] [hipe]
Elixir 1.7.2 (compiled with Erlang/OTP 19)

erl
Erlang/OTP 21 [erts-10.0.5] [source] [64-bit] [smp:4:4] [ds:4:4:10] [async-threads:1] [hipe]

/Users/dal/.asdf/installs/elixir/1.7.2/.mix/rebar --version
rebar 2.6.2 R15B03 20160628_123811 git 2.6.2

/Users/dal/.asdf/installs/elixir/1.7.2/.mix/rebar3 --version
rebar 3.6.1 on Erlang/OTP 21 Erts 10.0.5

I have these versions of deps

* certifi 2.4.2 (Hex package) (rebar3)
  locked at 2.4.2 (certifi) 75424ff0
  ok
* hackney 1.14.3 (Hex package) (rebar3)
  locked at 1.14.3 (hackney) b5f6f5dc
  ok

If there is something else I could provide, let me know.

[bugant]

@dalhorinek I noticed your elixir version has been compiled on Erlang/OTP 19 while you have Erlang/OTP 21 installed. Could that be the issue?

[dalhorinek]

@bugant I've just noticed it as well when I was reporting the versions, I've just upgraded to 1.7.4 which is compiled with 21 and tried it again. It seems that this is not the issue.

$ elixir -v                                                                                                                                                 ✔  12:07:00 08.11.18  INSERT
Erlang/OTP 21 [erts-10.0.5] [source] [64-bit] [smp:4:4] [ds:4:4:10] [async-threads:1] [hipe]

Elixir 1.7.4 (compiled with Erlang/OTP 21)

Here's how I'm able to reproduce it

$ rm -rf _build

$ mix deps.compile
===> Compiling parse_trans
===> Compiling mimerl
===> Compiling metrics
==> bunt
Compiling 2 files (.ex)
Generated bunt app
===> Compiling unicode_util_compat
===> Compiling idna
==> gen_stage
Compiling 10 files (.ex)
Generated gen_stage app
==> jason
Compiling 8 files (.ex)
Generated jason app
==> ssl_verify_fun
Compiling 7 files (.erl)
Generated ssl_verify_fun app
==> html_entities
Compiling 2 files (.ex)
Generated html_entities app
===> Compiling certifi
===> Compiling hackney
==> httpoison
Compiling 3 files (.ex)
Generated httpoison app
==> credo
Compiling 181 files (.ex)
Generated credo app
===> Compiling mochiweb
src/mochiweb_socket.erl:91: Warning: ssl:ssl_accept/2: deprecated; use ssl:handshake/2 instead

==> floki
Compiling 1 file (.erl)
Compiling 21 files (.ex)
Generated floki app

$ mix compile
Compiling 6 files (.ex)
Generated crawler app

$ iex -S mix
Erlang/OTP 21 [erts-10.0.5] [source] [64-bit] [smp:4:4] [ds:4:4:10] [async-threads:1] [hipe]

Interactive Elixir (1.7.4) - press Ctrl+C to exit (type h() ENTER for help)
iex(1)> :hackney.request(:get, "https://www.kolkovna.cz")
{:ok, 200,
 [
   {"Server", "openresty"},
   {"Date", "Thu, 08 Nov 2018 11:03:14 GMT"},
   {"Content-Type", "text/html; charset=utf-8"},
   {"Transfer-Encoding", "chunked"},
   {"Connection", "keep-alive"},
   {"X-Powered-By", "Nette Framework"},
   {"X-Frame-Options", "SAMEORIGIN"},
   {"Expires", "Thu, 19 Nov 1981 08:52:00 GMT"},
   {"Cache-Control",
    "no-store, no-cache, must-revalidate, post-check=0, pre-check=0"},
   {"Pragma", "no-cache"},
   {"Set-Cookie", "nette-browser=xzwed87n27; path=/; httponly"},
   {"Set-Cookie",
    "PHPSESSID=28d0fb01459c7c26ceeaff8942b3cf77; expires=Thu, 22-Nov-2018 11:03:14 GMT; Max-Age=1209600; path=/; HttpOnly"},
   {"Vary", "Accept-Encoding"}
 ], #Reference<0.2514075068.711720964.60255>}
iex(2)>
BREAK: (a)bort (c)ontinue (p)roc info (i)nfo (l)oaded
       (v)ersion (k)ill (D)b-tables (d)istribution

$ vim lib/crawler.ex # I updated just some text in IO.puts

$ iex -S mix
Erlang/OTP 21 [erts-10.0.5] [source] [64-bit] [smp:4:4] [ds:4:4:10] [async-threads:1] [hipe]

Interactive Elixir (1.7.4) - press Ctrl+C to exit (type h() ENTER for help)
iex(1)> :hackney.request(:get, "https://www.kolkovna.cz")
** (UndefinedFunctionError) function :certifi.cacerts/0 is undefined (module :certifi is not available)
    (certifi) :certifi.cacerts()
    (hackney) /Users/dal/Work/Analyticoo/crawler/deps/hackney/src/hackney_connect.erl:326: :hackney_connect.ssl_opts_1/2
    (hackney) /Users/dal/Work/Analyticoo/crawler/deps/hackney/src/hackney_connect.erl:273: :hackney_connect.do_connect/5
    (hackney) /Users/dal/Work/Analyticoo/crawler/deps/hackney/src/hackney_connect.erl:45: :hackney_connect.connect/5
    (hackney) /Users/dal/Work/Analyticoo/crawler/deps/hackney/src/hackney.erl:329: :hackney.request/5
iex(1)>

[benoitc]

hrm i didn’t reproduced it today. do you have rebar3 installed?

anyway i’m making a minimal app to test.

On Thursday, November 8, 2018, Dal Horinek notifications@github.com wrote:

@bugant https://github.com/bugant I've just noticed it as well when I was reporting the versions, I've just upgraded to 1.7.4 which is compiled with 21 and tried it again. It seems that this is not the issue.

$ elixir -v  ✔  12:07:00 08.11.18  INSERT Erlang/OTP 21 [erts-10.0.5] [source] [64-bit] [smp:4:4] [ds:4:4:10] [async-threads:1] [hipe]

Elixir 1.7.4 (compiled with Erlang/OTP 21)

Here's how I able to reproduce it

$ rm -rf _build

$ mix deps.compile ===> Compiling parse_trans ===> Compiling mimerl ===> Compiling metrics ==> bunt Compiling 2 files (.ex) Generated bunt app ===> Compiling unicode_util_compat ===> Compiling idna ==> gen_stage Compiling 10 files (.ex) Generated gen_stage app ==> jason Compiling 8 files (.ex) Generated jason app ==> ssl_verify_fun Compiling 7 files (.erl) Generated ssl_verify_fun app ==> html_entities Compiling 2 files (.ex) Generated html_entities app ===> Compiling certifi ===> Compiling hackney ==> httpoison Compiling 3 files (.ex) Generated httpoison app ==> credo Compiling 181 files (.ex) Generated credo app ===> Compiling mochiweb src/mochiweb_socket.erl:91: Warning: ssl:ssl_accept/2: deprecated; use ssl:handshake/2 instead

==> floki Compiling 1 file (.erl) Compiling 21 files (.ex) Generated floki app

$ mix compile Compiling 6 files (.ex) Generated crawler app

$ iex -S mix Erlang/OTP 21 [erts-10.0.5] [source] [64-bit] [smp:4:4] [ds:4:4:10] [async-threads:1] [hipe]

Interactive Elixir (1.7.4) - press Ctrl+C to exit (type h() ENTER for help) iex(1)> :hackney.request(:get, "https://www.kolkovna.cz") {:ok, 200, [ {"Server", "openresty"}, {"Date", "Thu, 08 Nov 2018 11:03:14 GMT"}, {"Content-Type", "text/html; charset=utf-8"}, {"Transfer-Encoding", "chunked"}, {"Connection", "keep-alive"}, {"X-Powered-By", "Nette Framework"}, {"X-Frame-Options", "SAMEORIGIN"}, {"Expires", "Thu, 19 Nov 1981 08:52:00 GMT"}, {"Cache-Control", "no-store, no-cache, must-revalidate, post-check=0, pre-check=0"}, {"Pragma", "no-cache"}, {"Set-Cookie", "nette-browser=xzwed87n27; path=/; httponly"}, {"Set-Cookie", "PHPSESSID=28d0fb01459c7c26ceeaff8942b3cf77; expires=Thu, 22-Nov-2018 11:03:14 GMT; Max-Age=1209600; path=/; HttpOnly"}, {"Vary", "Accept-Encoding"} ], #Reference<0.2514075068.711720964.60255>} iex(2)> BREAK: (a)bort (c)ontinue (p)roc info (i)nfo (l)oaded (v)ersion (k)ill (D)b-tables (d)istribution

$ vim lib/crawler.ex # I updated just some text in IO.puts

$ iex -S mix Erlang/OTP 21 [erts-10.0.5] [source] [64-bit] [smp:4:4] [ds:4:4:10] [async-threads:1] [hipe]

Interactive Elixir (1.7.4) - press Ctrl+C to exit (type h() ENTER for help) iex(1)> :hackney.request(:get, "https://www.kolkovna.cz") ** (UndefinedFunctionError) function :certifi.cacerts/0 is undefined (module :certifi is not available) (certifi) :certifi.cacerts() (hackney) /Users/dal/Work/Analyticoo/crawler/deps/hackney/src/hackney_connect.erl:326: :hackney_connect.ssl_opts_1/2 (hackney) /Users/dal/Work/Analyticoo/crawler/deps/hackney/src/hackney_connect.erl:273: :hackney_connect.do_connect/5 (hackney) /Users/dal/Work/Analyticoo/crawler/deps/hackney/src/hackney_connect.erl:45: :hackney_connect.connect/5 (hackney) /Users/dal/Work/Analyticoo/crawler/deps/hackney/src/hackney.erl:329: :hackney.request/5 iex(1)>

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/benoitc/hackney/issues/528#issuecomment-436959111, or mute the thread https://github.com/notifications/unsubscribe-auth/AAA4ovhOr7F2RrhVpRklmDQ43wRSqNbmks5utBFsgaJpZM4WkAaf .

-- Sent from my Mobile

[alg]

I'm also having this issue, but it seems it's not specific to certifi in my case. The other day I was "losing" Cowboy in the very same way -- just couldn't start the app because some Cowboy module was missing. I had to delete its directory in _build and then start the app. Upon restart had to repeat it.

[alg]

As a followup, by installing a dtrace probe I figured that it's beam.smp process that is deleting .beam files at some moments. I also noticed that it happens during recompilation when I change something in code.

I omitted many lines and posting just few of them to show which packages are affected. They seem to be the same every time.

  1    181                     unlink:entry beam.smp[30141] /Volumes/Data/Users/alg/p/deps/parse_trans/ebin/parse_trans_pp.beam
  2    181                     unlink:entry beam.smp[30158] /Volumes/Data/Users/alg/p/deps/mimerl/ebin/mimerl.beam
  1    181                     unlink:entry beam.smp[30226] /Volumes/Data/Users/alg/p/deps/metrics/ebin/metrics.beam
  1    181                     unlink:entry beam.smp[30231] /Volumes/Data/Users/alg/p/deps/unicode_util_compat/ebin/unicode_util_compat.beam
  0    181                     unlink:entry beam.smp[30322] /Volumes/Data/Users/alg/p/deps/certifi/ebin/certifi.beam
  1    181                     unlink:entry beam.smp[30236] /Volumes/Data/Users/alg/p/deps/idna/ebin/idna.beam
  1    181                     unlink:entry beam.smp[30242] /Volumes/Data/Users/alg/p/deps/ranch/ebin/ranch_conns_sup.beam

[benoitc]

@alg did you install rebar3 or just using mix that will fetch its own version?

[rupurt]

@alg great insight. Do you mind posting a gist of how you observed that? I'd like to help figure this out but don't know where to start.

[alg]

@alg did you install rebar3 or just using mix that will fetch its own version?

@benoitc I have rebar3 installed globally. Not sure which version is used by mix though.

@alg great insight. Do you mind posting a gist of how you observed that? I'd like to help figure this out but don't know where to start.

@rupurt Sure, put the following snippet into a file (say, trace.d) and then call $ sudo dtrace -s trace.d. This piece watches what the "beam.smp" process deletes. You can play around with dtrace options to get to a specific file, for example. Unfortunately, I wasn't able to get the stack trace so far as Mac has very strict memory access policy:

#!/usr/sbin/dtrace -wqs

syscall::unlink:entry
/execname == "beam.smp"/
{
  trace(cleanpath(copyinstr(arg0)));
}

I discovered which process is removing certifi.beam file by using the probe:

#!/usr/sbin/dtrace -wqs

syscall::unlink:entry
/cleanpath(copyinstr(arg0)) == "/Volumes/Data/Users/alg/project/deps/certifi/ebin/certifi.beam"/
{
  trace(execname);
}

[alg]

The curious thing is that I picked another Elixir project I worked on a bit ago and you know what? It suffers from the same issue and if I run mix deps.compile several times in a row, it compiles exactly the same set of dependencies. In my case they are (in this order):

===> Compiling parse_trans
===> Compiling mimerl
===> Compiling metrics
===> Compiling unicode_util_compat
===> Compiling idna
===> Compiling ranch
===> Compiling poolboy
===> Compiling ssl_verify_fun
==> bcrypt_elixir
===> Compiling certifi
===> Compiling hackney
===> Compiling cowlib
===> Compiling cowboy
==> phoenix

[benoitc]

@alg you only get the error with certifi?

[benoitc]

what would be interresting to test to make sure it's not an issue with mix compilation would be testing the following cases:

• an app that only use certifi
• an app that is using hackney + certifi

also related to benoitc/erlang-idna#26 an app that is only using idna. I will try later today to do it but if someone can also do it on its side, that would be really appreciated.

[alg]

@benoitc I started a new empty mix project and added certifi to it. Repeated mix deps.compile resulted in repeated compilation of parse_trans and certifi. Then I replaced certifi with explicit parse_trans 3.3.0. Now continuous recompilation of parse_trans.

Just to be sure I'm not going crazy and it shouldn't be like that, I added jsx 2.8 and ecto 3.0.1. After the initial compilation, repeated deps compilations triggered only recompilation of again... parse_trans. See below:

alg:~/tmp/certifi_only
 → mix deps.get
Resolving Hex dependencies...
Dependency resolution completed:
Unchanged:
  jsx 2.9.0
  parse_trans 3.3.0
New:
  decimal 1.6.0
  ecto 3.0.4
* Getting ecto (Hex package)
* Getting decimal (Hex package)

alg:~/tmp/certifi_only
 → mix deps.compile
===> Compiling parse_trans
==> decimal
Compiling 1 file (.ex)
Generated decimal app
==> ecto
Compiling 54 files (.ex)
Generated ecto app

alg:~/tmp/certifi_only
 → mix deps.compile
===> Compiling parse_trans

I'm going to try and figure what is it with parse_trans that it triggers this recompilation.

[alg]

A bit more progress. Lift your hands who use Vim with Ale plugin.

Even when I disabled its Elixir linter, it still triggered the recompilation in the background (that's how it gets error messages for display) and that ruined .beam files (I even noticed several *.crashdump files in deps). So I had to disable it entirely.

It still doesn't explain continuous recompilation, but at least (as it looks after a few minutes of playing around) .beam files don't disappear now.

[benoitc]

I guess there is something with the way elixir is behaving with parse_transform. I may revert to the old code that generating the erlang file first to see if it's the root cause.

[benoitc]

@alg thanks for the debugging btw. that helps a lot :)

[dalhorinek]

I use vim with Ale, so this is possible.

Here is an output of ALEInfo after some code change.

(finished - exit code 1) ['/bin/zsh', '-c', 'cd ''/Users/dal/projects/title_fetch'' && MIX_BUILD_PATH=''/var/folders/w9/6bs426js14g39x9wvx2xncth0000gn/T/nvimJCqYKd/268'' mix compile ''/Users/dal/projects/title_fetch/lib/title_fetch.ex''']
<<<OUTPUT STARTS>>>
===> Compiling parse_trans
===> Compiling mimerl
===> Compiling metrics
===> Compiling unicode_util_compat
===> Compiling idna
===> Compiling ssl_verify_fun
==> html_entities
Compiling 2 files (.ex)
Generated html_entities app
===> Compiling certifi
===> Compiling src/certifi.erl failed
src/certifi.erl:none: undefined parse transform 'ct_expand'
==> title_fetch
<<<OUTPUT ENDS>>>

And those number directories (268) are always created and then after some time removed ... it contains something like this

$ find 268 -depth 2
268/lib/metrics
268/lib/parse_trans
268/lib/ssl_verify_fun
268/lib/idna
268/lib/unicode_util_compat
268/lib/mimerl

So it compiles it into the temporary folder, then removes it, so that should not affect the build in main directory?

Could that whole issue be caused by the error there?

src/certifi.erl:none: undefined parse transform 'ct_expand'

[dalhorinek]

I can confirm that's ALE for sure. When I disable ALE using :ALEDisable I'm not able to reproduce it anymore the way I was able to reproduce it. When I enable it again :ALEEnable, the problem is back on.

@alg Nice catch!

[alg]

@dalhorinek I also noticed that vim process is deleting files from some temp directory, but... before that beam.smp is removing exactly the same files from the deps. It gives me creeps actually. :)

So the issue forked into two different:

• some libraries keep recompiling slowing things down on start/restart quite a bit (I look at you, parse_trans and co)
• ALE misbehaving. I guess, this one is better be discussed there in ale's repo.