Internal error struct undefined

[tcoopman]

version: 0.6.0

code:

defmodule EventstoreSqliteTest do
  use ExUnit.Case

  use Mneme
  use EventstoreSqlite.DataCase
  use TypedStruct

  doctest EventstoreSqlite

  typedstruct module: FooTestEvent do
    field(:text, :string)
  end

  typedstruct module: Complex do
    field(:c, :string)
  end

  typedstruct module: ComplexEvent do
    field(:complex, Complex.t())
  end

....

    test "$all stream" do
      event_1 = %FooTestEvent{text: "1"}
      event_2 = %FooTestEvent{text: "2"}
      event_3 = %FooTestEvent{text: "3"}
      :ok = EventstoreSqlite.append_to_stream("stream-1", [event_1, event_2, event_3])
      :ok = EventstoreSqlite.append_to_stream("stream-2", [event_2, event_3])

      auto_assert(
        [
          %EventstoreSqlite.RecordedEvent{
            data: %FooTestEvent{text: "3"},
            type: "Elixir.EventstoreSqliteTest.FooTestEvent"
          },
          %EventstoreSqlite.RecordedEvent{
            data: %FooTestEvent{text: "2"},
            type: "Elixir.EventstoreSqliteTest.FooTestEvent"
          },
          %EventstoreSqlite.RecordedEvent{
            data: %FooTestEvent{text: "3"},
            type: "Elixir.EventstoreSqliteTest.FooTestEvent"
          },
          %EventstoreSqlite.RecordedEvent{
            data: %FooTestEvent{text: "2"},
            type: "Elixir.EventstoreSqliteTest.FooTestEvent"
          },
          %EventstoreSqlite.RecordedEvent{
            data: %FooTestEvent{text: "1"},
            type: "Elixir.EventstoreSqliteTest.FooTestEvent"
          }
        ] <- EventstoreSqlite.read_stream_backward("$all")
      )
    end

error:

 1) test read_stream_backward $all stream (EventstoreSqliteTest)
     test/eventstore_sqlite_test.exs:533
     ** (Mneme.InternalError) Mneme encountered an internal error. This is likely a bug in Mneme.

     Please consider reporting this error at https://github.com/zachallaun/mneme/issues. Thanks!

     ** (UndefinedFunctionError) function Complex.__struct__/0 is undefined (module Complex is not available)
         Complex.__struct__()
         (mneme 0.6.0) lib/mneme/assertion/pattern_builder.ex:221: Mneme.Assertion.PatternBuilder.struct_to_patterns/5
         (mneme 0.6.0) lib/mneme/assertion/pattern_builder.ex:26: Mneme.Assertion.PatternBuilder.to_patterns/3
         (mneme 0.6.0) lib/mneme/assertion/pattern_builder.ex:187: anonymous fn/3 in Mneme.Assertion.PatternBuilder.enumerate_map_patterns/3
         (elixir 1.16.2) lib/enum.ex:1301: anonymous fn/3 in Enum.flat_map_reduce/3
         (elixir 1.16.2) lib/enum.ex:4839: Enumerable.List.reduce/3
         (elixir 1.16.2) lib/enum.ex:1300: Enum.flat_map_reduce/3
         (mneme 0.6.0) lib/mneme/assertion/pattern_builder.ex:184: Mneme.Assertion.PatternBuilder.enumerate_map_patterns/3
         (elixir 1.16.2) lib/enum.ex:1301: anonymous fn/3 in Enum.flat_map_reduce/3
         (elixir 1.16.2) lib/enum.ex:4839: Enumerable.List.reduce/3
         (elixir 1.16.2) lib/enum.ex:1300: Enum.flat_map_reduce/3
         (mneme 0.6.0) lib/mneme/assertion/pattern_builder.ex:26: Mneme.Assertion.PatternBuilder.to_patterns/3
         (mneme 0.6.0) lib/mneme/assertion/pattern_builder.ex:226: Mneme.Assertion.PatternBuilder.struct_to_patterns/5
         (mneme 0.6.0) lib/mneme/assertion/pattern_builder.ex:26: Mneme.Assertion.PatternBuilder.to_patterns/3
         (elixir 1.16.2) lib/enum.ex:1826: Enum."-map_reduce/3-lists^mapfoldl/2-0-"/3
         (elixir 1.16.2) lib/enum.ex:1826: Enum."-map_reduce/3-lists^mapfoldl/2-0-"/3
         (mneme 0.6.0) lib/mneme/assertion/pattern_builder.ex:246: Mneme.Assertion.PatternBuilder.enum_to_patterns/3
         (mneme 0.6.0) lib/mneme/assertion/pattern_builder.ex:94: Mneme.Assertion.PatternBuilder.do_to_patterns/3
         (mneme 0.6.0) lib/mneme/assertion/pattern_builder.ex:26: Mneme.Assertion.PatternBuilder.to_patterns/3
         (mneme 0.6.0) lib/mneme/assertion/pattern_builder.ex:21: Mneme.Assertion.PatternBuilder.to_patterns/2

     code: auto_assert(
     stacktrace:
       (mneme 0.6.0) lib/mneme/assertion.ex:193: Mneme.Assertion.handle_assertion/4
       (mneme 0.6.0) lib/mneme/assertion.ex:152: Mneme.Assertion.do_run/3
       (mneme 0.6.0) lib/mneme/assertion.ex:133: Mneme.Assertion.run/3
       test/eventstore_sqlite_test.exs:540: (test)

..

The weird thing is that other tests that seem to do similar things in the file succeed. But it's not at random, the same tests always fail, also when run in isolation.

If I delete the pattern so mneme has to create a new one, mneme stil fails.

Notice that the Complex struct mentioned in the error (UndefinedFunctionError) function Complex.__struct__/0 is undefined (module Complex is not available) is nowhere to be found in the code of the test though...

There was a time when this worked - but it also fails on 0.5.0 now that I test it. And I'm confused, because I believe it used to work on that version...

[zachallaun]

Is there any chance you can create a single failing test file that can be run standalone using Mix.install()? The general format is this:

# example_test.exs
Mix.install([
  {:mneme, "0.6.0"},
  ...
])

ExUnit.start()
Mneme.start()

defmodule ExampleTest do
  use ExUnit.Case
  use Mneme

  # test case that causes internal error
  ...
end

ExUnit.run()

If all is set up correctly, you should just be able to run elixir example_test.exs and see the crash. This should make it possible for me to reproduce and fix. Thanks!

[tcoopman]

I'll try again later, but couldn't reproduce it yet.

If it helps you: The repo is here btw: https://github.com/tcoopman/eventstore_sqlite

link to a failing test: https://github.com/tcoopman/eventstore_sqlite/blob/943fe83134e36c998408c628fc59ba7244c60f83/test/eventstore_sqlite_test.exs#L318

[zachallaun]

Can you share the Elixir and OTP versions you're using?

[tcoopman]

 elixir --version
Erlang/OTP 26 [erts-14.2.4] [source] [64-bit] [smp:32:32] [ds:32:32:10] [async-threads:1] [jit:ns]

Elixir 1.16.2 (compiled with Erlang/OTP 26)

[tcoopman]

The change 1.15 to 1.16 could be the issue

[zachallaun]

This error is quite baffling. So I found an old discussion about a somewhat similar issue that pointed to an explanation that José wrote, but I'm not sure it applies here.

The relevant lines that led up to the crash essentially look like:

defp do_to_patterns(%struct{} = value, context, vars) do
  ...
  struct_to_patterns(struct, value, context, vars, [])
end

defp struct_to_patterns(struct, value, context, vars, notes) do
  # crash
  defaults = struct.__struct__()
  ...
end

So it's possible that, in some weird code-path, a %Complex{} (which desugars to %{__struct__: Complex, ...}) was returned by a function without the Complex module actually being loaded.

[tcoopman]

Well, now I'm super confused.

I've just changed my test to:

defmodule EventstoreSqliteTest do
  use ExUnit.Case

  use Mneme
  use EventstoreSqlite.DataCase
  use TypedStruct

  doctest EventstoreSqlite

  typedstruct module: FooTestEvent do
    field(:text, :string)
  end

  # typedstruct module: Complex2 do
  #   field(:c, :string)
  # end

  # typedstruct module: ComplexEvent do
  #   field(:complex2, Complex2.t())
  # end

  describe "append_to_stream/2" do
    test "$all stream" do
      event_1 = %FooTestEvent{text: "1"}
      event_2 = %FooTestEvent{text: "2"}
      event_3 = %FooTestEvent{text: "3"}
      :ok = EventstoreSqlite.append_to_stream("stream-1", [event_1, event_2, event_3])
      :ok = EventstoreSqlite.append_to_stream("stream-2", [event_2, event_3])

      auto_assert(
        [
          %EventstoreSqlite.RecordedEvent{
            data: %FooTestEvent{text: "3"},
            type: "Elixir.EventstoreSqliteTest.FooTestEvent"
          },
          %EventstoreSqlite.RecordedEvent{
            data: %FooTestEvent{text: "2"},
            type: "Elixir.EventstoreSqliteTest.FooTestEvent"
          },
          %EventstoreSqlite.RecordedEvent{
            data: %FooTestEvent{text: "3"},
            type: "Elixir.EventstoreSqliteTest.FooTestEvent"
          },
          %EventstoreSqlite.RecordedEvent{
            data: %FooTestEvent{text: "2"},
            type: "Elixir.EventstoreSqliteTest.FooTestEvent"
          },
          %EventstoreSqlite.RecordedEvent{
            data: %FooTestEvent{text: "1"},
            type: "Elixir.EventstoreSqliteTest.FooTestEvent"
          }
        ] <- EventstoreSqlite.read_stream_backward("$all")
      )
    end
  end
end

and removed _build, run the test again and I get the exact same failure. About module Complex not being available.

Where is it coming from??? I don't have any Complex modules anymore in my code

[tcoopman]

I've found it. Stupid me.

I've ran a benchmark against the test database and also loaded a Complex module (but a different one) in the db. The benchmark didn't clean up the database, so the tests were loading an incorrect module that didn't exist...

Sorry for the trouble

[zachallaun]

Really weird. How are you running the test? mix test test/eventstore_sqlite_test.exs?

[zachallaun]

Ah! Makes sense. No worries at all and thanks for continuing to dig into it!

This does give me some ideas though, because it's possible to trigger this with:

defmodule Repro do
  use ExUnit.Case
  use Mneme

  test "repro" do
    auto_assert(%{__struct__: Foo})
  end
end

So minimally, I should be guarding against this!