Errors occure from time to time running unit tests

[VeljkoMaksimovic]

Elixir and Erlang/OTP versions

Erlang/OTP 24 [erts-12.3.2] [source] [64-bit] [smp:1:1] [ds:1:1:10] [async-threads:1]

Elixir 1.12.1 (compiled with Erlang/OTP 24)

Operating system

Had this issuee on Ubuntu18.04 and macOS Monterey 12.1

Current behavior

This error occurres from time to time when running unit tests. If I run same test again, it all works good.

def(test read_user_permissions/1 When user_id isnt given(_)) do
  (
    mock_modules = Enum.reduce([{Guard.Rbac.RoleBindingIdentification, [], [fetch: fn _, :user_id -> {:ok, nil} end]}, {Cacheman, [], [get: fn :guard_rbac_test, key -> {:ok, <<String.Chars.to_string(key)::binary(), "_p1,", String.Chars.to_string(key)::binary(), "_p2">>} end]}], [], fn {m, opts, mock_fns}, ms ->
      case(Enum.member?(ms, m)) do
        x when :erlang.orelse(:erlang."=:="(x, false), :erlang."=:="(x, nil)) ->
          try do
            case(:meck.validate(m)) do
              x when :erlang.orelse(:erlang."=:="(x, false), :erlang."=:="(x, nil)) ->
                nil
              _ ->
                :meck.unload(m)
            end
          rescue
            e in [ErlangError] ->
              :ok
          end
          :meck.new(m, opts)
        _ ->
          nil
      end
      Mock._install_mock(m, mock_fns)
      true = :meck.validate(m)
      Enum.uniq([m | ms])
    end)
    try do
      (
        left = :erlang.element(1, Guard.Rbac.Cache.read_user_permissions(%Guard.Rbac.RoleBindingIdentification{org_id: nil, project_id: nil, user_id: nil}))
        right = :error
        ExUnit.Assertions.assert(:erlang.==(left, right), left: left, right: right, expr: {:assert, [line: 67], [{:==, [line: 67], [{:|>, [line: 67], [{{:., [line: 67], [{:__aliases__, [line: 67], [:Cache]}, :read_user_permissions]}, [line: 67], [{:%, [line: 67], [{:__aliases__, [line: 67], [:RBI]}, {:%{}, [line: 67], []}]}]}, {:elem, [line: 67], [0]}]}, :error]}]}, message: "Assertion with == failed", context: :==)
      )
      (
        unquoted_module = Cacheman
        unquoted_f = :get
        unquoted_args = [:guard_rbac_test, :_]
        unquoted_call_times = 0
        num_calls = :meck.num_calls(unquoted_module, unquoted_f, unquoted_args)
        case(:erlang."/="(num_calls, unquoted_call_times)) do
          false ->
            nil
          true ->
            mfa_str = <<String.Chars.to_string(unquoted_module)::binary(), ".", String.Chars.to_string(unquoted_f)::binary(), "(", Enum.join(Enum.map(unquoted_args, &Kernel.inspect/1), ", ")::binary(), ")">>
            :erlang.error(ExUnit.AssertionError.exception(message: <<"Expected ", String.Chars.to_string(mfa_str)::binary(), " to be called exactly ", String.Chars.to_string(unquoted_call_times)::binary(), " time(s), but it was called (number of calls: ", String.Chars.to_string(num_calls)::binary(), ")">>))
        end
      )
    after
      for(m <- mock_modules) do
        :meck.unload(m)
      end
    end
  )
  :ok
end

Please report this bug: https://github.com/elixir-lang/elixir/issues

** (UndefinedFunctionError) function Guard.Rbac.RoleBindingIdentification.__struct__/0 is undefined or private
        (guard 0.1.0) Guard.Rbac.RoleBindingIdentification.__struct__()
        (elixir 1.12.1) lib/module/types/of.ex:131: Module.Types.Of.struct/3
        (elixir 1.12.1) lib/module/types/expr.ex:193: Module.Types.Expr.of_expr/4
        (elixir 1.12.1) lib/module/types/helpers.ex:93: Module.Types.Helpers.do_map_reduce_ok/3
        (elixir 1.12.1) lib/module/types/expr.ex:379: Module.Types.Expr.of_expr/4
        (elixir 1.12.1) lib/module/types/helpers.ex:93: Module.Types.Helpers.do_map_reduce_ok/3
        (elixir 1.12.1) lib/module/types/expr.ex:379: Module.Types.Expr.of_expr/4
        (elixir 1.12.1) lib/module/types/expr.ex:140: Module.Types.Expr.of_expr/4
        (elixir 1.12.1) lib/module/types/helpers.ex:93: Module.Types.Helpers.do_map_reduce_ok/3
        (elixir 1.12.1) lib/module/types/expr.ex:209: Module.Types.Expr.of_expr/4
        (elixir 1.12.1) lib/module/types/helpers.ex:93: Module.Types.Helpers.do_map_reduce_ok/3
        (elixir 1.12.1) lib/module/types/expr.ex:209: Module.Types.Expr.of_expr/4
        (elixir 1.12.1) lib/module/types/expr.ex:511: Module.Types.Expr.of_expr_context/4
        (elixir 1.12.1) lib/module/types/expr.ex:265: anonymous fn/3 in Module.Types.Expr.of_expr/4
        (elixir 1.12.1) lib/module/types/helpers.ex:37: Module.Types.Helpers.do_reduce_ok/3
        (elixir 1.12.1) lib/module/types/expr.ex:246: Module.Types.Expr.of_expr/4
        (elixir 1.12.1) lib/module/types/helpers.ex:93: Module.Types.Helpers.do_map_reduce_ok/3
        (elixir 1.12.1) lib/module/types/expr.ex:209: Module.Types.Expr.of_expr/4
        (elixir 1.12.1) lib/module/types/helpers.ex:93: Module.Types.Helpers.do_map_reduce_ok/3
        (elixir 1.12.1) lib/module/types/expr.ex:209: Module.Types.Expr.of_expr/4

Expected behavior

Running same code works ~ 9 out of 10 times

[josevalim]

We have fixed bugs related to this in the past. Please update to Elixir v1.13.4 and let us know if it continues. Thank you!

[VeljkoMaksimovic]

@josevalim Updated to version 1.13.4

Erlang/OTP 24 [erts-12.3.2] [source] [64-bit] [smp:1:1] [ds:1:1:10] [async-threads:1]

Elixir 1.13.4 (compiled with Erlang/OTP 23)

Error still persists. I ran same test script 15 times, 2 passes had this error, and other 13 times all tests executed as expected.

The script contains 7 tests, and its not the case that error occurres always during the execution of same test. It is always triggered by different tests.

[josevalim]

Thank you! How are you running tests? Are you loading multiple files or a single file?

[VeljkoMaksimovic]

Single file with command mix test _path_to_test_file_

[josevalim]

Is it async: true or async: false? If the former, does it error with async: false?

[VeljkoMaksimovic]

I was running it asynchronously. When I switch to async: false everything is working fine, and I cant reproduce the error.

[josevalim]

Ok, I know what is happening: :meck replaces a module globally and while the test is running (async: true starts running it as soon as it is defined), the compiler is doing code verification. However, the compiler cannot find the struct definition because :meck temporarily removed or replaced the module.

You should use async: false or use something else for mocking, because :meckis literally not safe for concurrent usage.

Edit: also see #11218.