UndefinedFunctionError: function <Struct>.__struct__/0 is undefined or private

[feifanzhou]

Environment

• Elixir & Erlang/OTP versions (elixir --version): Erlang/OTP 24 [erts-12.0.3] [source] [64-bit] [smp:24:24] [ds:24:24:10] [async-threads:1] [jit] [dtrace] Elixir 1.12.2 (compiled with Erlang/OTP 24)
• Operating system: macOS 11.5.1 (20G80)
• Editor: VSCode 1.59.1 with ElixirLS 0.8.0 running continuously
• Tests using mock 0.3.7

Current behavior

Hello! I'm getting the error ** (UndefinedFunctionError) function B2.RequestContext.__struct__/0 is undefined or private when mix test gets to a particular ExUnit test that references the module in the error (see below for the full text of the error and source of the struct and test file).

Unfortunately, I'm not sure how specifically to induce the problem, although it does happen almost all the time. Additional behaviors I notice:

• This error happens on tests annotated with @tag :skip
• This crash happens every time I run mix test to run my entire test suite.
• It does not crash when I run mix test filename:linenumber to run just the specific test identified in the error.
• If I comment out this particular test, running the full mix test will occasionally pass, but usually crash with the same error on a different test that references this struct.
• mix clean sometimes makes the problem go away on the next run of mix test, but other times the crash will still happen after mix clean.

Expected behavior

This crash shouldn't happen; the __struct__/0 function should be defined.

Appendix

I'm happy to add people to my private repo with the full project if that helps repo. But to start, here's the error message and relevant code:

Full error from running mix test:

11:11:23.919 [error] Process #PID<0.1990.0> raised an exception
** (Module.Types.Error) found error while checking types for Tanagram.Schema.FileTypeTest."test FileItemCreationRequest returns stored values for b2_* fields for B2-backend requests"/1

def(test FileItemCreationRequest returns stored values for b2_* fields for B2-backend requests(_)) do
  (
    user = Tanagram.Factory.insert(:user)
    hours_24_from_now = DateTime.add(DateTime.utc_now(), :erlang.*(3600, 24), :second)
    query = "mutation {\n  createFileItemCreationRequest(backend: B2) {\n    b2_bucket_id\n    b2_authorization_token\n  }\n}\n"
    (
      mock_modules = Enum.reduce([{B2.RequestContext, [], [from_key: fn _, _ -> {:ok, %B2.RequestContext{api_key: nil, api_key_id: nil, authorization_context: nil, authorization_header: nil, base_url: nil, version: "2"}} end]}], [], fn {m, opts, mock_fns}, ms ->
        case(Enum.member?(ms, m)) do
          x when :erlang.orelse(:erlang."=:="(x, false), :erlang."=:="(x, nil)) ->
            try do
              case(:meck.validate(m)) do
                x when :erlang.orelse(:erlang."=:="(x, false), :erlang."=:="(x, nil)) ->
                  nil
                _ ->
                  :meck.unload(m)
              end
            rescue
              e in [ErlangError] ->
                :ok
            end
            :meck.new(m, opts)
          _ ->
            nil
        end
        Mock._install_mock(m, mock_fns)
        true = :meck.validate(m)
        Enum.uniq([m | ms])
      end)
      try do
        mock_modules = Enum.reduce([{B2.Api.GetUploadUrl, [], [call: fn _, _ -> {:ok, %{bucket_id: "test_bucket_id", upload_url: "https://pod-000-1005-03.backblaze.com/test_upload_url", authorization_token: "test_upload_auth_token", expires_at: hours_24_from_now}} end]}], [], fn {m, opts, mock_fns}, ms ->
          case(Enum.member?(ms, m)) do
            x when :erlang.orelse(:erlang."=:="(x, false), :erlang."=:="(x, nil)) ->
              try do
                case(:meck.validate(m)) do
                  x when :erlang.orelse(:erlang."=:="(x, false), :erlang."=:="(x, nil)) ->
                    nil
                  _ ->
                    :meck.unload(m)
                end
              rescue
                e in [ErlangError] ->
                  :ok
              end
              :meck.new(m, opts)
            _ ->
              nil
          end
          Mock._install_mock(m, mock_fns)
          true = :meck.validate(m)
          Enum.uniq([m | ms])
        end)
        try do
          (
            right = Absinthe.run(query, Tanagram.Schema, context: %{current_user: user})
            expr = {:assert, [line: 66], [{:=, [line: 66], [{:ok, {:%{}, [line: 59], [data: {:%{}, [line: 60], [{"createFileItemCreationRequest", {:%{}, [line: 61], [{"b2_authorization_token", {:auth_token, [line: 62], nil}}, {"b2_bucket_id", {:bucket_id, [line: 63], nil}}]}}]}]}}, {{:., [line: 66], [{:__aliases__, [line: 66], [:Absinthe]}, :run]}, [line: 66], [{:query, [line: 66], nil}, {:__aliases__, [line: 66], [:Tanagram, :Schema]}, [context: {:%{}, [line: 66], [current_user: {:user, [line: 66], nil}]}]]}]}]}
            [bucket_id, auth_token] = case(right) do
              {:ok, %{data: %{"createFileItemCreationRequest" => %{"b2_authorization_token" => auth_token, "b2_bucket_id" => bucket_id}}}} ->
                case(right) do
                  x when :erlang.orelse(:erlang."=:="(x, nil), :erlang."=:="(x, false)) ->
                    :erlang.error(ExUnit.AssertionError.exception(expr: expr, message: <<"Expected truthy, got ", Kernel.inspect(right)::binary()>>))
                  _ ->
                    :ok
                end
                [bucket_id, auth_token]
              _ ->
                left = {:ok, {:%{}, [line: 59], [data: {:%{}, [line: 60], [{"createFileItemCreationRequest", {:%{}, [line: 61], [{"b2_authorization_token", {:auth_token, [line: 62], nil}}, {"b2_bucket_id", {:bucket_id, [line: 63], nil}}]}}]}]}}
                :erlang.error(ExUnit.AssertionError.exception(left: left, right: right, expr: expr, message: <<"match (=) failed", ExUnit.Assertions.__pins__([])::binary()>>, context: {:match, []}))
            end
            right
          )
          (
            left = auth_token
            right = "test_upload_auth_token"
            ExUnit.Assertions.assert(:erlang.==(left, right), left: left, right: right, expr: {:assert, [line: 68], [{:==, [line: 68], [{:auth_token, [line: 68], nil}, "test_upload_auth_token"]}]}, message: "Assertion with == failed", context: :==)
          )
          (
            left = bucket_id
            right = "test_bucket_id"
            ExUnit.Assertions.assert(:erlang.==(left, right), left: left, right: right, expr: {:assert, [line: 69], [{:==, [line: 69], [{:bucket_id, [line: 69], nil}, "test_bucket_id"]}]}, message: "Assertion with == failed", context: :==)
          )
        after
          for(m <- mock_modules) do
            :meck.unload(m)
          end
        end
      after
        for(m <- mock_modules) do
          :meck.unload(m)
        end
      end
    )
  )
  :ok
end

Please report this bug: https://github.com/elixir-lang/elixir/issues

** (UndefinedFunctionError) function B2.RequestContext.__struct__/0 is undefined (module B2.RequestContext is not available)
    (tanagram 0.1.0) B2.RequestContext.__struct__()
    (elixir 1.12.2) lib/module/types/of.ex:131: Module.Types.Of.struct/3
    (elixir 1.12.2) lib/module/types/expr.ex:193: Module.Types.Expr.of_expr/4
    (elixir 1.12.2) lib/module/types/helpers.ex:93: Module.Types.Helpers.do_map_reduce_ok/3
    (elixir 1.12.2) lib/module/types/expr.ex:125: Module.Types.Expr.of_expr/4
    (elixir 1.12.2) lib/module/types/expr.ex:482: anonymous fn/3 in Module.Types.Expr.of_clauses/3
    (elixir 1.12.2) lib/module/types/helpers.ex:37: Module.Types.Helpers.do_reduce_ok/3
    (elixir 1.12.2) lib/module/types/expr.ex:232: Module.Types.Expr.of_expr/4
    (elixir 1.12.2) lib/module/types/helpers.ex:93: Module.Types.Helpers.do_map_reduce_ok/3
    (elixir 1.12.2) lib/module/types/expr.ex:125: Module.Types.Expr.of_expr/4
    (elixir 1.12.2) lib/module/types/helpers.ex:93: Module.Types.Helpers.do_map_reduce_ok/3
    (elixir 1.12.2) lib/module/types/expr.ex:82: Module.Types.Expr.of_expr/4
    (elixir 1.12.2) lib/module/types/helpers.ex:93: Module.Types.Helpers.do_map_reduce_ok/3
    (elixir 1.12.2) lib/module/types/expr.ex:125: Module.Types.Expr.of_expr/4
    (elixir 1.12.2) lib/module/types/helpers.ex:93: Module.Types.Helpers.do_map_reduce_ok/3
    (elixir 1.12.2) lib/module/types/expr.ex:82: Module.Types.Expr.of_expr/4
    (elixir 1.12.2) lib/module/types/helpers.ex:93: Module.Types.Helpers.do_map_reduce_ok/3
    (elixir 1.12.2) lib/module/types/expr.ex:379: Module.Types.Expr.of_expr/4
    (elixir 1.12.2) lib/module/types/expr.ex:140: Module.Types.Expr.of_expr/4
    (elixir 1.12.2) lib/module/types/helpers.ex:93: Module.Types.Helpers.do_map_reduce_ok/3

Struct module source:

defmodule B2.RequestContext do
  @type t :: %__MODULE__{
          api_key_id: String.t(),
          api_key: String.t(),
          authorization_context: %B2.AuthorizationContext{} | nil,
          authorization_header: String.t(),
          base_url: String.t(),
          version: String.t()
        }
  defstruct(
    api_key_id: nil,
    api_key: nil,
    authorization_context: nil,
    authorization_header: nil,
    base_url: nil,
    version: "2"
  )

  def with_auth_header(%__MODULE__{api_key_id: nil} = context), do: context
  def with_auth_header(%__MODULE__{api_key_id: ""} = context), do: context
  def with_auth_header(%__MODULE__{api_key: nil} = context), do: context
  def with_auth_header(%__MODULE__{api_key: ""} = context), do: context

  def with_auth_header(%__MODULE__{api_key_id: api_key_id, api_key: api_key} = context) do
    header = "Basic " <> Base.encode64("#{api_key_id}:#{api_key}")
    %{context | authorization_header: header}
  end

  def from_key(api_key_id, api_key) do
    module = %__MODULE__{api_key_id: api_key_id, api_key: api_key} |> with_auth_header()

    case B2.Api.AuthorizeAccount.call(module) do
      {:ok, auth_context} ->
        {:ok, %{module | authorization_context: auth_context, base_url: auth_context.api_url}}

      error ->
        error
    end
  end
end

Test code source:

defmodule Tanagram.Schema.FileTypeTest do
  use Tanagram.DataCase, async: true
  import Mock

  @tag :skip
  describe "FileItemCreationRequest" do
    test "returns nil for b2_* fields for local-backend requests" do
      user = insert(:user)

      query = """
      mutation {
        createFileItemCreationRequest(backend: LOCAL) {
          b2_bucket_id
          b2_authorization_token
        }
      }
      """

      assert {:ok,
              %{
                data: %{
                  "createFileItemCreationRequest" => %{
                    "b2_authorization_token" => auth_token,
                    "b2_bucket_id" => bucket_id
                  }
                }
              }} = Absinthe.run(query, Tanagram.Schema, context: %{current_user: user})

      assert is_nil(auth_token)
      assert is_nil(bucket_id)
    end

    @tag :skip
    test "returns stored values for b2_* fields for B2-backend requests" do
      user = insert(:user)
      hours_24_from_now = DateTime.utc_now() |> DateTime.add(3600 * 24, :second)

      query = """
      mutation {
        createFileItemCreationRequest(backend: B2) {
          b2_bucket_id
          b2_authorization_token
        }
      }
      """

      with_mock B2.RequestContext, from_key: fn _, _ -> {:ok, %B2.RequestContext{}} end do
        with_mock B2.Api.GetUploadUrl,
          call: fn _, _ ->
            {:ok,
             %{
               bucket_id: "test_bucket_id",
               upload_url: "https://pod-000-1005-03.backblaze.com/test_upload_url",
               authorization_token: "test_upload_auth_token",
               expires_at: hours_24_from_now
             }}
          end do
          assert {:ok,
                  %{
                    data: %{
                      "createFileItemCreationRequest" => %{
                        "b2_authorization_token" => auth_token,
                        "b2_bucket_id" => bucket_id
                      }
                    }
                  }} = Absinthe.run(query, Tanagram.Schema, context: %{current_user: user})

          assert auth_token == "test_upload_auth_token"
          assert bucket_id == "test_bucket_id"
        end
      end
    end
  end
end

[josevalim]

I am suspecting this is a race condition in your tests.

with_mock is most likely replacing your module B2.RequestContext and, while it is being replaced, it affects how other modules that invoke B2.RequestContext are compiled. Does the issue happens if you set async: false on all tests that mock this struct or if you do mix test --max-cases 1?

[feifanzhou]

Thank you José! That does appear to be the problem — toggling async on my tests makes the problem go away or come back. It hadn't occurred to me that async tests would still share a single compiled "instance" of a particular module. Are there any plans to increase isolation when running async tests, such that these tests could run concurrently?

[josevalim]

@feifanzhou the thing to keep in mind is that ExUnit runs async tests while test files are still loading. And then the mock library is globally replacing a module while that happens. You should either use a mock library that is safe under concurrency (such as Mox) or run tests synchronously.

[josevalim]

Closing this, as there is nothing the compiler can do if something is changing modules globally. :) Thank you!