Private hex registry error on Hex > 2.1.0

Hi, in our company we are currently using Hex repositories hosted locally in a private data center.

After upgrading the hex tool to version 2.1.0 / 2.1.1, we do not have the ability to download private packages from other repositories besides the main hex.pm repo.

We observed the same errors in:

repositories built with the official hex tool (https://hex.pm/docs/self-hosting)
repositories built with Artipie (https://github.com/artipie/artipie with hexpm adapter)
repositories built with hex_core (version 0.10.3) & our custom elixir code

Steps to reproduce

Prepare env (env can be created with asdf, classic dpkg, or Docker)

Hex:    2.1.1 (or 2.1.0)
Elixir: 1.17.2 (but this error is valid on 1.16.X)
OTP:    27.0.1 (this error is valid also on 26.2.X)

and download latest hex

$ mix local.hex

Create local hex repository with hex tool (https://hex.pm/docs/self-hosting)

$ mkdir hex_repo && cd hex_repo;
$ openssl genrsa -out repo.pem 4096
$ mix hex.registry build public --name=locker_hex --private-key=repo.pem

Create sample package

$ mix new sample

with mix.exs

defmodule Sample.MixProject do
  use Mix.Project

  def project do
    [
      app: :sample,
      version: "0.1.0",
      elixir: "~> 1.17",
      start_permanent: Mix.env() == :prod,
      description: "Example lib to locally test the local hex app",
      package: package(),
      deps: deps()
    ]
  end

  # Run "mix help compile.app" to learn about applications.
  def application do
    [
      extra_applications: [:logger]
    ]
  end

  # Run "mix help deps" to learn about dependencies.
  defp deps do
    [
      {:ex_doc, ">= 0.30.0", only: :dev, runtime: false}
    ]
  end

  defp package do
    [
      licenses: ["Apache 2.0"],
      links: %{}
    ]
  end
end

Build it for publication

$ mix hex.build

$ cp sample-0.1.0.tar ../hex_repo/public/tarballs

Update repo

$ mix hex.registry build public --name=locker_hex --private-key=repo.pem

* updating public/packages/sample
* updating public/names
* updating public/versions

Host it with php -S localhost:8000 or provided Erlang code erl -s inets -eval 'inets:start(httpd,[{port,8000},{server_name,"localhost"},{server_root,"."},{document_root,"public"}]).' or python simple server etc.
Add repo to local hex

$ mix hex.repo add locker_hex http://localhost:8000 --public-key=hex_repo/public/public_key

Try to fetch published package

mix hex.package fetch sample 0.1.0 --repo=locker_hex

Errors

With no HEX_NO_VERIFY_REPO_ORIGIN env

21:28:00.079 [error] GenServer Hex.Registry.Server terminating
** (Mix.Error) Fetched deprecated registry record version from repo locker_hex. For security reasons this registry version is no longer supported. The repository you are using should update to fix the security reason. Set HEX_NO_VERIFY_REPO_ORIGIN=1 to disable this check.
    (mix 1.16.2) lib/mix.ex:580: Mix.raise/2
    (hex 2.0.6) lib/hex/registry/server.ex:354: Hex.Registry.Server.write_result/4
    (hex 2.0.6) lib/hex/registry/server.ex:218: Hex.Registry.Server.handle_info/2
    (stdlib 5.2) gen_server.erl:1095: :gen_server.try_handle_info/3
    (stdlib 5.2) gen_server.erl:1183: :gen_server.handle_msg/6
    (stdlib 5.2) proc_lib.erl:241: :proc_lib.init_p_do_apply/3
# (...)

With HEX_NO_VERIFY_REPO_ORIGIN=1

21:28:57.481 [error] GenServer Hex.Registry.Server terminating
** (FunctionClauseError) no function clause matching in anonymous fn/1 in Hex.Registry.Server.write_result/4
    (hex 2.1.1) lib/hex/registry/server.ex:367: anonymous fn({:name, "sample"}) in Hex.Registry.Server.write_result/4
    (elixir 1.17.2) lib/enum.ex:992: anonymous fn/3 in Enum.each/2
    (stdlib 6.0.1) maps.erl:860: :maps.fold_1/4
    (elixir 1.17.2) lib/enum.ex:2543: Enum.each/2
    (hex 2.1.1) lib/hex/registry/server.ex:367: Hex.Registry.Server.write_result/4
    (hex 2.1.1) lib/hex/registry/server.ex:226: Hex.Registry.Server.handle_info/2
    (stdlib 6.0.1) gen_server.erl:2173: :gen_server.try_handle_info/3
    (stdlib 6.0.1) gen_server.erl:2261: :gen_server.handle_msg/6
Last message: {:get_package, "locker_hex", "sample", {:ok, {200, <<31, 139, 8, 0, 0, 0, 0, 0, 0, 19, 1, 98, 2, 157, 253, 10, 93, 10, 75, 10, 5, 48, 46, 49, 46, 48, 18, 32, 60, 124, 11, 185, 133, 56, 11, 122, 190, 62, 85, 252, 138, 252, ...>>, %{"access-control-allow-credentials" => "true", "access-control-allow-origin" => "*", "access-control-expose-headers" => "", "cache-control" => "max-age=0, private, must-revalidate", "content-length" => "633", "content-type" => "application/vnd.hex+erlang; charset=utf-8", "date" => "Tue, 13 Aug 2024 19:28:56 GMT", "vary" => "accept-encoding", "x-request-id" => "F-tgdN4gG68VjFQAAAPH"}}}}
State: %{ets: #Reference<0.2403431140.535166980.117764>, pending: MapSet.new([{"locker_hex", "sample"}]), waiting: %{{"locker_hex", "sample"} => [{{#PID<0.94.0>, [:alias | #Reference<0.0.12035.2403431140.535101444.117951>]}, #Function<5.67092853/0 in Hex.Registry.Server.handle_call/3>}]}, path: "/Users/[redacted]/.hex/cache.ets", fetched: MapSet.new([]), pending_fun: nil}

Error from our custom registry with hex_core lib: same

Error from Artipie: https://github.com/artipie/artipie/issues/1464

Current workaround

Downgrade hex to 2.0.0-2.0.6

$ mix local.hex 2.0.6 --force

* creating /Users/[redacted]/.asdf/installs/elixir/1.17.2-otp-27/.mix/archives/hex-2.0.6

This can be related to: https://github.com/hexpm/hex/issues/1029

With hex 2.0.6 downloading deps from private repos is working without issue.

$ mix hex.package fetch sample 0.1.0 --repo=locker_hex

sample v0.1.0 downloaded to /Users/[redacted]/testflight/sample-0.1.0.tar

Classic project with mix.exs:

defmodule Whatever.MixProject do
  use Mix.Project

  def project do
    [
      app: :whatever,
      version: "0.1.0",
      elixir: "~> 1.14",
      start_permanent: Mix.env() == :prod,
      deps: deps(),
      hex: hex()
    ]
  end

  # Run "mix help compile.app" to learn about applications.
  def application do
    [
      extra_applications: [:logger]
    ]
  end

  # Run "mix help deps" to learn about dependencies.
  defp deps do
    [
      # {:dep_from_hexpm, "~> 0.3.0"},
      # {:dep_from_git, git: "https://github.com/elixir-lang/my_dep.git", tag: "0.1.0"}
      {:sample, "~> 0.1.0", repo: :locker_hex}
    ]
  end

  defp hex() do
    [
      # unsafe_registry: true,
      no_verify_repo_origin: true
    ]
  end
end

works too with mix deps.get

hexpm / hex