scoutapp / scout_apm_elixir

ScoutAPM Elixir Agent. Supports Phoenix and other frameworks.
https://scoutapm.com
Other
36 stars 20 forks source link

Application stops tracking when upgrading to v0.3.1 #31

Closed tcollier closed 7 years ago

tcollier commented 7 years ago

After upgrading the elixir scout_apm from v0.3.0 to v0.3.1, my Scout dashboard reports no requests, despite server logs showing a steady stream of requests.

Here is a screen shot of the dashboard for an external monitoring request that happens every minute. The drop in throughput corresponds to the deploy that upgraded scout_apm.

screen shot 2017-06-07 at 11 50 11 am

The server logs show the following error

Last message: :check_if_time_to_update
18:38:14.387 [info] Setup ScoutApm.Watcher on ScoutApm.Store
18:38:14.387 [info] Setup ScoutApm.Watcher on ScoutApm.Config
18:38:14.387 [info] Setup ScoutApm.Watcher on ScoutApm.PersistentHistogram
18:38:14.387 [info] Setup ScoutApm.Watcher on ScoutApm.Logger
18:38:14.387 [info] Setup ScoutApm.Watcher on ScoutApm.Supervisor
18:38:14.418 [info] ScoutAPM Started
18:38:14.806 [error] GenServer ScoutApm.ApplicationLoadNotification terminating
** (UndefinedFunctionError) function :unicode_util.lowercase/1 is undefined (module :unicode_util is not available)
    :unicode_util.lowercase('checkin.scoutapp.com')
    (idna) /tmp/build_48b567470daa6b1447c11058fc2e02de/deps/idna/src/idna.erl:57: :idna.lowercase_list/1
    (idna) /tmp/build_48b567470daa6b1447c11058fc2e02de/deps/idna/src/idna.erl:10: :idna.to_ascii/1
    (hackney) /tmp/build_48b567470daa6b1447c11058fc2e02de/deps/hackney/src/hackney_url.erl:96: :hackney_url.normalize/2
    (hackney) /tmp/build_48b567470daa6b1447c11058fc2e02de/deps/hackney/src/hackney.erl:291: :hackney.request/5
    (scout_apm) lib/scout_apm/application_load_notification.ex:97: ScoutApm.ApplicationLoadNotification.post/1
    (scout_apm) lib/scout_apm/application_load_notification.ex:37: ScoutApm.ApplicationLoadNotification.handle_cast/2
Last message: {:"$gen_cast", {:run, [retries: 3]}}
    (stdlib) gen_server.erl:601: :gen_server.try_dispatch/4
18:38:15.721 [info] Running SixPack.Endpoint with Cowboy using http://localhost:31438
18:38:17.194 [error] GenServer :tzdata_release_updater terminating
    :unicode_util.lowercase('www.iana.org')
    (idna) /tmp/build_48b567470daa6b1447c11058fc2e02de/deps/idna/src/idna.erl:57: :idna.lowercase_list/1
** (UndefinedFunctionError) function :unicode_util.lowercase/1 is undefined (module :unicode_util is not available)
    (idna) /tmp/build_48b567470daa6b1447c11058fc2e02de/deps/idna/src/idna.erl:10: :idna.to_ascii/1
    (hackney) /tmp/build_48b567470daa6b1447c11058fc2e02de/deps/hackney/src/hackney_url.erl:96: :hackney_url.normalize/2
    (hackney) /tmp/build_48b567470daa6b1447c11058fc2e02de/deps/hackney/src/hackney.erl:291: :hackney.request/5
    (tzdata) lib/tzdata/data_loader.ex:42: Tzdata.DataLoader.last_modified_of_latest_available/1
    (tzdata) lib/tzdata/release_updater.ex:64: Tzdata.ReleaseUpdater.loaded_tzdata_matches_remote_last_modified?/0
    (tzdata) lib/tzdata/release_updater.ex:36: Tzdata.ReleaseUpdater.poll_for_update/0
Last message: :check_if_time_to_update
tcollier commented 7 years ago

My application is running on Heroku

% uname -a
Linux d1864176-6470-48f0-8b9a-de1b7466eb22 3.13.0-112-generic #159-Ubuntu SMP Fri Mar 3 15:26:07 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux
% iex --version
Erlang/OTP 19 [erts-8.3] [source] [64-bit] [smp:8:8] [ds:8:8:10] [async-threads:10] [hipe] [kernel-poll:false]

IEx 1.4.2

And has the following mix.lock

%{"approximate_histogram": {:hex, :approximate_histogram, "0.1.1", "198eb36681e763ed4baab6ca0682acec4ef642f60ba272f251d3059052f4f378", [:mix], [], "hexpm"},
  "certifi": {:hex, :certifi, "1.2.1", "c3904f192bd5284e5b13f20db3ceac9626e14eeacfbb492e19583cf0e37b22be", [:rebar3], [], "hexpm"},
  "combine": {:hex, :combine, "0.9.6", "8d1034a127d4cbf6924c8a5010d3534d958085575fa4d9b878f200d79ac78335", [:mix], [], "hexpm"},
  "connection": {:hex, :connection, "1.0.4", "a1cae72211f0eef17705aaededacac3eb30e6625b04a6117c1b2db6ace7d5976", [:mix], [], "hexpm"},
  "cowboy": {:hex, :cowboy, "1.1.2", "61ac29ea970389a88eca5a65601460162d370a70018afe6f949a29dca91f3bb0", [:rebar3], [{:cowlib, "~> 1.0.2", [hex: :cowlib, repo: "hexpm", optional: false]}, {:ranch, "~> 1.3.2", [hex: :ranch, repo: "hexpm", optional: false]}], "hexpm"},
  "cowlib": {:hex, :cowlib, "1.0.2", "9d769a1d062c9c3ac753096f868ca121e2730b9a377de23dec0f7e08b1df84ee", [:make], [], "hexpm"},
  "db_connection": {:hex, :db_connection, "1.1.2", "2865c2a4bae0714e2213a0ce60a1b12d76a6efba0c51fbda59c9ab8d1accc7a8", [:mix], [{:connection, "~> 1.0.2", [hex: :connection, repo: "hexpm", optional: false]}, {:poolboy, "~> 1.5", [hex: :poolboy, repo: "hexpm", optional: true]}, {:sbroker, "~> 1.0", [hex: :sbroker, repo: "hexpm", optional: true]}], "hexpm"},
  "decimal": {:hex, :decimal, "1.3.1", "157b3cedb2bfcb5359372a7766dd7a41091ad34578296e951f58a946fcab49c6", [:mix], [], "hexpm"},
  "decorators": {:hex, :decorators, "0.1.0", "1f4fd3682de23c6bce769201613812f5eaf809310a27e35ddbfd91e53b126267", [:rebar3], [{:parse_trans, "2.9.0", [hex: :parse_trans, repo: "hexpm", optional: false]}], "hexpm"},
  "ecto": {:hex, :ecto, "2.1.4", "d1ba932813ec0e0d9db481ef2c17777f1cefb11fc90fa7c142ff354972dfba7e", [:mix], [{:db_connection, "~> 1.1", [hex: :db_connection, repo: "hexpm", optional: true]}, {:decimal, "~> 1.2", [hex: :decimal, repo: "hexpm", optional: false]}, {:mariaex, "~> 0.8.0", [hex: :mariaex, repo: "hexpm", optional: true]}, {:poison, "~> 2.2 or ~> 3.0", [hex: :poison, repo: "hexpm", optional: true]}, {:poolboy, "~> 1.5", [hex: :poolboy, repo: "hexpm", optional: false]}, {:postgrex, "~> 0.13.0", [hex: :postgrex, repo: "hexpm", optional: true]}, {:sbroker, "~> 1.0", [hex: :sbroker, repo: "hexpm", optional: true]}], "hexpm"},
  "edown": {:hex, :edown, "0.8.1", "7333b6f6b7bbc736c263e9ceb8261667d7c8f7d92e251829f1a9fe8f24e7b694", [:rebar3], [], "hexpm"},
  "ex_machina": {:hex, :ex_machina, "2.0.0", "ec284c6f57233729cea9319e083f66e613e82549f78eccdb2059aeba5d0df9f3", [:mix], [{:ecto, "~> 2.1", [hex: :ecto, repo: "hexpm", optional: true]}], "hexpm"},
  "gettext": {:hex, :gettext, "0.13.1", "5e0daf4e7636d771c4c71ad5f3f53ba09a9ae5c250e1ab9c42ba9edccc476263", [:mix], [], "hexpm"},
  "hackney": {:hex, :hackney, "1.8.5", "56025cf08aaf0998a5ad6494888a069fc3800f60eb6bdd1fca7adf0f1329c888", [:rebar3], [{:certifi, "1.2.1", [hex: :certifi, repo: "hexpm", optional: false]}, {:idna, "5.0.2", [hex: :idna, repo: "hexpm", optional: false]}, {:metrics, "1.0.1", [hex: :metrics, repo: "hexpm", optional: false]}, {:mimerl, "1.0.2", [hex: :mimerl, repo: "hexpm", optional: false]}, {:ssl_verify_fun, "1.1.1", [hex: :ssl_verify_fun, repo: "hexpm", optional: false]}], "hexpm"},
  "idna": {:hex, :idna, "5.0.2", "ac203208ada855d95dc591a764b6e87259cb0e2a364218f215ad662daa8cd6b4", [:rebar3], [{:unicode_util_compat, "0.2.0", [hex: :unicode_util_compat, repo: "hexpm", optional: false]}], "hexpm"},
  "jiffy": {:hex, :jiffy, "0.14.7", "9f33b893edd6041ceae03bc1e50b412e858cc80b46f3d7535a7a9940a79a1c37", [:make, :rebar], [], "hexpm"},
  "lhttpc": {:hex, :lhttpc, "1.4.0", "61760afec1ddb98e47972be93b13fbff487a63d65b91a02c41122a0adf83da38", [:rebar3], [], "hexpm"},
  "meck": {:hex, :meck, "0.8.4", "59ca1cd971372aa223138efcf9b29475bde299e1953046a0c727184790ab1520", [:make, :rebar], [], "hexpm"},
  "metrics": {:hex, :metrics, "1.0.1", "25f094dea2cda98213cecc3aeff09e940299d950904393b2a29d191c346a8486", [:rebar3], [], "hexpm"},
  "mime": {:hex, :mime, "1.1.0", "01c1d6f4083d8aa5c7b8c246ade95139620ef8effb009edde934e0ec3b28090a", [:mix], [], "hexpm"},
  "mimerl": {:hex, :mimerl, "1.0.2", "993f9b0e084083405ed8252b99460c4f0563e41729ab42d9074fd5e52439be88", [:rebar3], [], "hexpm"},
  "mock": {:hex, :mock, "0.2.1", "bfdba786903e77f9c18772dee472d020ceb8ef000783e737725a4c8f54ad28ec", [:mix], [{:meck, "~> 0.8.2", [hex: :meck, repo: "hexpm", optional: false]}], "hexpm"},
  "new_relixir": {:git, "https://github.com/apartmentlist/new-relixir.git", "30da3d430a5c5bcedd20b5627df4475d76e9180a", [tag: "v0.2.1.al"]},
  "newrelic": {:hex, :newrelic, "0.1.0", "dd80750994644701fabb99fe7531e3bb6d200957cd1cb5c5f455ed7f9e6d0f90", [:rebar3], [{:jiffy, "0.14.7", [hex: :jiffy, repo: "hexpm", optional: false]}, {:lhttpc, "1.4.0", [hex: :lhttpc, repo: "hexpm", optional: false]}, {:statman, "0.5.0", [hex: :statman, repo: "hexpm", optional: false]}], "hexpm"},
  "parse_trans": {:hex, :parse_trans, "2.9.0", "3f5f7b402928fb9fd200c891e635de909045d1efac40ce3f924d3892898f85eb", [:rebar], [{:edown, "> 0.0.0", [hex: :edown, repo: "hexpm", optional: false]}], "hexpm"},
  "phoenix": {:hex, :phoenix, "1.2.4", "4172479b5e21806a5e4175b54820c239e0d4effb0b07912e631aa31213a05bae", [:mix], [{:cowboy, "~> 1.0", [hex: :cowboy, repo: "hexpm", optional: true]}, {:phoenix_pubsub, "~> 1.0", [hex: :phoenix_pubsub, repo: "hexpm", optional: false]}, {:plug, "~> 1.4 or ~> 1.3.3 or ~> 1.2.4 or ~> 1.1.8 or ~> 1.0.5", [hex: :plug, repo: "hexpm", optional: false]}, {:poison, "~> 1.5 or ~> 2.0", [hex: :poison, repo: "hexpm", optional: false]}], "hexpm"},
  "phoenix_ecto": {:hex, :phoenix_ecto, "3.2.3", "450c749876ff1de4a78fdb305a142a76817c77a1cd79aeca29e5fc9a6c630b26", [:mix], [{:ecto, "~> 2.1", [hex: :ecto, repo: "hexpm", optional: false]}, {:phoenix_html, "~> 2.9", [hex: :phoenix_html, repo: "hexpm", optional: true]}, {:plug, "~> 1.0", [hex: :plug, repo: "hexpm", optional: false]}], "hexpm"},
  "phoenix_pubsub": {:hex, :phoenix_pubsub, "1.0.1", "c10ddf6237007c804bf2b8f3c4d5b99009b42eca3a0dfac04ea2d8001186056a", [:mix], [], "hexpm"},
  "plug": {:hex, :plug, "1.3.5", "7503bfcd7091df2a9761ef8cecea666d1f2cc454cbbaf0afa0b6e259203b7031", [:mix], [{:cowboy, "~> 1.0.1 or ~> 1.1", [hex: :cowboy, repo: "hexpm", optional: true]}, {:mime, "~> 1.0", [hex: :mime, repo: "hexpm", optional: false]}], "hexpm"},
  "poison": {:hex, :poison, "2.2.0", "4763b69a8a77bd77d26f477d196428b741261a761257ff1cf92753a0d4d24a63", [:mix], [], "hexpm"},
  "poolboy": {:hex, :poolboy, "1.5.1", "6b46163901cfd0a1b43d692657ed9d7e599853b3b21b95ae5ae0a777cf9b6ca8", [:rebar], [], "hexpm"},
  "postgrex": {:hex, :postgrex, "0.13.3", "c277cfb2a9c5034d445a722494c13359e361d344ef6f25d604c2353185682bfc", [:mix], [{:connection, "~> 1.0", [hex: :connection, repo: "hexpm", optional: false]}, {:db_connection, "~> 1.1", [hex: :db_connection, repo: "hexpm", optional: false]}, {:decimal, "~> 1.0", [hex: :decimal, repo: "hexpm", optional: false]}], "hexpm"},
  "ranch": {:hex, :ranch, "1.3.2", "e4965a144dc9fbe70e5c077c65e73c57165416a901bd02ea899cfd95aa890986", [:rebar3], [], "hexpm"},
  "scout_apm": {:hex, :scout_apm, "0.3.1", "3b38681840c227b409169902b65ec7a5c7b1e1be6792f425100796aca7142b41", [:mix], [{:approximate_histogram, "~>0.1.1", [hex: :approximate_histogram, repo: "hexpm", optional: false]}, {:hackney, "~> 1.0", [hex: :hackney, repo: "hexpm", optional: false]}, {:plug, "~>1.0", [hex: :plug, repo: "hexpm", optional: false]}, {:poison, ">= 0.0.0", [hex: :poison, repo: "hexpm", optional: false]}], "hexpm"},
  "ssl_verify_fun": {:hex, :ssl_verify_fun, "1.1.1", "28a4d65b7f59893bc2c7de786dec1e1555bd742d336043fe644ae956c3497fbe", [:make, :rebar], [], "hexpm"},
  "statman": {:hex, :statman, "0.5.0", "3b130ed01bca478d34631752f678dc43ffe09d46d2e526554e285673c3497270", [:rebar3], [{:decorators, "0.1.0", [hex: :decorators, repo: "hexpm", optional: false]}], "hexpm"},
  "timex": {:hex, :timex, "3.1.15", "94abaec8fef2436ced4d0e1b4ed50c8eaa5fb9138fc0699946ddee7abf5aaff2", [:mix], [{:combine, "~> 0.7", [hex: :combine, repo: "hexpm", optional: false]}, {:gettext, "~> 0.10", [hex: :gettext, repo: "hexpm", optional: false]}, {:tzdata, "~> 0.1.8 or ~> 0.5", [hex: :tzdata, repo: "hexpm", optional: false]}], "hexpm"},
  "tzdata": {:hex, :tzdata, "0.5.12", "1c17b68692c6ba5b6ab15db3d64cc8baa0f182043d5ae9d4b6d35d70af76f67b", [:mix], [{:hackney, "~> 1.0", [hex: :hackney, repo: "hexpm", optional: false]}], "hexpm"},
  "unicode_util_compat": {:hex, :unicode_util_compat, "0.2.0", "dbbccf6781821b1c0701845eaf966c9b6d83d7c3bfc65ca2b78b88b8678bfa35", [:rebar3], [], "hexpm"}}
cschneid commented 7 years ago

This appears to be an issue with the newest hackney version.

https://github.com/benoitc/hackney/issues/413

If you don't need the latest version of hackney for your own app, you may be able to downgrade hackney by explicitly adding it to your mix deps. The bulk of the development and testing of the agent has been against hackney version 1.7.1.

tcollier commented 7 years ago

I locked my version of hackney to 1.8.3 and am now getting data in Scout. Thanks!