Runtime error while checking types on elixir 1.17 / OTP 27

[navinpeiris]

Elixir and Erlang/OTP versions

Erlang/OTP 27 [erts-15.0] [source] [64-bit] [smp:16:16] [ds:16:16:10] [async-threads:1]

Elixir 1.17.0 (compiled with Erlang/OTP 27)

Operating system

MacOS Sonoma 14.5 (Intel)

Current behavior

After upgrading to 1.17 / OTP 27, I get the following error while running tests. I've reduced the test to a bare date comparison to make it easier:

Test case:

test "date" do
  date = %Date{year: 2024, month: 2, day: 21}

  assert date.year == 2024
end

Console output:

(EXIT from #PID<0.94.0>) an exception was raised:
    ** (RuntimeError) found error while checking types for RedactedName."test something date"/1:

** (Protocol.UndefinedError) protocol Enumerable not implemented for nil of type Atom. This protocol is implemented for the following type(s): DBConnection.PrepareStream, DBConnection.Stream, Date.Range, Ecto.Adapters.SQL.Stream, File.Stream, Floki.HTMLTree, Function, GenEvent.Stream, HashDict, HashSet, IO.Stream, Jason.OrderedObject, List, Map, MapSet, Phoenix.LiveView.LiveStream, Postgrex.Stream, Range, Req.Response.Async, Stream, Timex.Interval
The exception happened while checking this code:

def test something date(_) do
  (
    date = %Date{calendar: Calendar.ISO, year: 2024, month: 2, day: 21}

    (
      left = date.year
      right = 2024

      ExUnit.Assertions.assert(:erlang.==(left, right),
        left: left,
        right: right,
        expr:
          {:assert, [line: 84],
           [
             {:==, [line: 84],
              [
                {{:., [line: 84], [{:date, [line: 84], nil}, :year]}, [no_parens: true, line: 84],
                 []},
                2024
              ]}
           ]},
        message: "Assertion with == failed",
        context: :==
      )
    )
  )

  :ok
end

Please report this bug at: https://github.com/elixir-lang/elixir/issues

        (elixir 1.17.0) lib/enum.ex:1: Enumerable.impl_for!/1
        (elixir 1.17.0) lib/enum.ex:166: Enumerable.reduce/3
        (elixir 1.17.0) lib/enum.ex:4423: Enum.reduce/3
        (elixir 1.17.0) lib/module/types/of.ex:182: Module.Types.Of.struct/6
        (elixir 1.17.0) lib/module/types/expr.ex:125: Module.Types.Expr.of_expr/3
        (elixir 1.17.0) lib/module/types/helpers.ex:128: Module.Types.Helpers.do_map_reduce_ok/3
        (elixir 1.17.0) lib/module/types/expr.ex:185: Module.Types.Expr.of_expr/3
        (elixir 1.17.0) lib/module/types/helpers.ex:128: Module.Types.Helpers.do_map_reduce_ok/3
        (elixir 1.17.0) lib/module/types/expr.ex:185: Module.Types.Expr.of_expr/3
        (elixir 1.17.0) lib/module/types.ex:56: Module.Types.warnings_from_clause/6
        (elixir 1.17.0) lib/module/types.ex:15: anonymous fn/7 in Module.Types.warnings/5
        (elixir 1.17.0) lib/enum.ex:4353: Enum.flat_map_list/2
        (elixir 1.17.0) lib/enum.ex:4354: Enum.flat_map_list/2
        (elixir 1.17.0) lib/module/parallel_checker.ex:264: Module.ParallelChecker.check_module/3
        (elixir 1.17.0) lib/module/parallel_checker.ex:82: anonymous fn/6 in Module.ParallelChecker.spawn/4

mix test --seed 454319  29.58s user 2.01s system 647% cpu 4.879 total

Expected behavior

The tests to run without throwing a runtime error

[josevalim]

Hi @navinpeiris! Unfortunately I could not reproduce it. Here is what I did:

1. mix new foo

2. Change the test file to have this:

   test "date" do
   date = %Date{year: 2024, month: 2, day: 21}
   
   assert date.year == 2024
   end

3. mix test

And then everything worked. The only way this would fail is if Date is not really a struct (perhaps an alias) or if it was a struct at some point but it is not a struct now. But I assume in your case it is always a struct. If you place IO.inspect Date.__info__(:struct) outside of the test case (in the module body), what does it return? What about IO.inspect :code.which(Date)? Could perhaps the Date struct being redefined?

[navinpeiris]

Thanks for all that info @josevalim!

It seems like this error is tied to using Mimic, which I use for stubs. I've raised an issue in that repo (auto-linked above) and created a minimal project that reproduced the issue: navinpeiris/mimic-elixir-1.17-issue.

Please feel free to close this issue if you don't feel any changes are needed in the Elixir repo.

Thanks a ton!

[josevalim]

It may be that mimic is replacing modules and replacing basic interfaces Elixir requires for modules to work? Let's see what comes from upstream indeed. Thank you for the follow up!

[harrisi]

It seems like 5cbea017 is the bad commit. For some reason, seemingly randomly, Date.__info__(:struct) returns nil. I don't know if it's relevant, but the issue only seems to occur when running async: true is set for ExUnit.Case. I don't know why that would matter.

One other odd thing is that :code.which(struct) returns []. I think that's due to it being an empty charlist, maybe?

For testing, I cloned the repo mentioned, ran it a few times to find a seed that fails, and set the seed for ExUnit. In my case it is seed: 116846, but I'm not sure if that would translate to other machines in a useful way. I checked out 5cbea017 and added the following to struct/6. git diff:

diff --git a/lib/elixir/lib/module/types/of.ex b/lib/elixir/lib/module/types/of.ex
index e3466a128..f781ab9c0 100644
--- a/lib/elixir/lib/module/types/of.ex
+++ b/lib/elixir/lib/module/types/of.ex
@@ -169,6 +169,13 @@ def struct({:%, meta, _}, struct, args_types, default_handling, stack, context)
     context = remote(struct, :__struct__, 0, meta, stack, context)
     term = term()

+    unless struct.__info__(:struct) do
+      IO.inspect(struct, label: "struct")
+      IO.inspect(struct.__info__(:struct), label: "struct.__info__(:struct)")
+      IO.inspect(:code.which(struct), label: ":code.which(struct)")
+      IO.inspect(struct.module_info(), label: "module_info()")
+    end
+
     defaults =
       for %{field: field} <- struct.__info__(:struct), field != :__struct__ do
         {field, term}

The output I get is:

Running ExUnit with seed: 116846, max_cases: 16

........................struct: Date
.struct.__info__(:struct): nil
......:code.which(struct): []
.............................module_info(): [
  module: Date,
  exports: [
    __mimic_info__: 0,
    __info__: 1,
    __duration__!: 1,
    __struct__: 0,
    __struct__: 1,
    add: 2,
    after?: 2,
    before?: 2,
    beginning_of_month: 1,
    beginning_of_week: 1,
    beginning_of_week: 2,
    compare: 2,
    convert: 2,
    convert!: 2,
    day_of_era: 1,
    day_of_week: 1,
    day_of_week: 2,
    day_of_year: 1,
    days_in_month: 1,
    diff: 2,
    end_of_month: 1,
    end_of_week: 1,
    end_of_week: 2,
    from_erl: 1,
    from_erl: 2,
    from_erl!: 1,
    from_erl!: 2,
    from_gregorian_days: 1,
    from_gregorian_days: 2,
    from_iso8601: 1,
    from_iso8601: 2,
    from_iso8601!: 1,
    from_iso8601!: 2,
    leap_year?: 1,
    months_in_year: 1,
    new: 3,
    new: 4,
    new!: 3,
    new!: 4,
    quarter_of_year: 1,
    range: 2,
    range: 3,
    shift: 2,
    to_erl: 1,
    to_gregorian_days: 1,
    to_iso8601: 1,
    to_iso8601: 2,
    to_iso_days: 1,
    ...
  ],
  attributes: [vsn: [34970559376529178790725801832244147558]],
  compile: [
    version: ~c"8.4.3",
    options: [:no_spawn_compiler_process, :from_core, :no_core_prepare,
     :no_auto_import],
    source: ~c"/Users/ian/dev/navinpeiris/mimic-elixir-1.17-issue/deps/mimic/lib/mimic/module.ex"
  ],
  md5: <<26, 79, 21, 178, 3, 127, 110, 131, 42, 84, 153, 235, 79, 21, 117, 102>>
]
..........................................................** (EXIT from #PID<0.98.0>) an exception was raised:
    ** (RuntimeError) found error while checking types for HelloWorldTest."test test today 26"/1:
# ...

Not sure if any of that helps.

[josevalim]

It only happens on async: true because Mimic is replacing the module while another test file is being loaded and compiling (hence the checker is running). You can try running a single test with Mimic and checking if Date.__info__(:struct) and if :code.which/1 returns empty, if they do, then Mimic is most likely the root cause for changing the module behaviour.

[harrisi]

Ah, right. They do return nil and [] after the call to expect/4. I think that fully puts this into the realm of Mimic. Thanks, I'll move over to that discussion!

[navinpeiris]

Can confirm that this issue was fixed in the Mimic repo https://github.com/edgurgel/mimic/issues/65. Closing this issue, thanks everyone for all the help!