mix: dependency issue with optional dependencies and umbrellas

[SteffenDE]

Elixir and Erlang/OTP versions

Elixir 1.17.0-dev (4631784) (compiled with Erlang/OTP 26)

Operating system

macOS

Current behavior

mix phx.new test --database sqlite3 --umbrella
cd test_umbrella
rm -rf deps
mix deps.get --only e2e

...
* Updating phoenix_live_view (Hex package)
* Updating castore (Hex package)
* Getting floki (Hex package)
** (exit) exited in: GenServer.call(:hex_fetcher, {:await, {:tarball, "hexpm", "floki", "0.36.0"}}, 120000)
    ** (EXIT) time out
    (elixir 1.16.1) lib/gen_server.ex:1114: GenServer.call/3
    (hex 2.0.6) lib/hex/scm.ex:128: Hex.SCM.update/1
    (hex 2.0.6) lib/hex/scm.ex:227: Hex.SCM.checkout/1
    (mix 1.16.1) lib/mix/dep/fetcher.ex:64: Mix.Dep.Fetcher.do_fetch/3
    (mix 1.16.1) lib/mix/dep/converger.ex:229: Mix.Dep.Converger.all/8
    (mix 1.16.1) lib/mix/dep/converger.ex:162: Mix.Dep.Converger.init_all/8
    (mix 1.16.1) lib/mix/dep/converger.ex:146: Mix.Dep.Converger.all/4
    (mix 1.16.1) lib/mix/dep/converger.ex:89: Mix.Dep.Converger.converge/4

Another minimal reproduction is here https://github.com/SteffenDE/mix_optional_deps_umbrella_bug

The structure: We have an umbrella application app_umbrella that depends on dependency b dep_b in its application my_app, but also includes dep_b as dependency in the umbrella root. dep_b depends on dep_a with optional: true. my_app depends on dep_a with only: :test. The issue occurs when fetching dependencies for another env, for example mix deps.get --only e2e.

In a real application dep_b would be phoenix_live_view and dep_a floki.

Expected behavior

It should work, so either

mix should not include optional dependencies of apps added in the umbrella root if they are not required for an env given with --only.

or

hex shouldn't fail.

Please feel free to close this if it's not a mix, but a hex issue instead.

[josevalim]

here is a smaller repro: https://github.com/josevalim/test_umbrella

[SteffenDE]

Thank you very much for the quick fix. Will this be backported to a future 1.16 release?

[josevalim]

Unfortunately it won’t. Changes to the converger are too delicate to backport. Maybe it works if you also add Floki as only test at the root? It may also work if you use LV as a git dep for now.

[SteffenDE]

All good, we'll use only: [:test, :e2e] for floki until Elixir 1.17 is out.