Calling `stub_with/2` with a second module that doesn't exist causes Mimic to fail in unpredictable ways

[apreifsteck]

It's late, so I haven't crafted a reproducer, but I renamed a module that was being stubbed elsewhere in my code and didn't change it there. Then a bunch of tests failed saying that other completely unrelated modules hadn't been copied, when they definitely had. Other times the Mimic genserver crashed outright.

[edgurgel]

Thanks for the issue report! Yeah we should definitely not fail like that 👍

[jimsynz]

I tried to reproduce this, but it fails with UndefinedFunctionError which I would expect:

09:01:52.581 [error] GenServer Mimic.Server terminating
** (UndefinedFunctionError) function MartyMcFly.module_info/0 is undefined (module MartyMcFly is not available)
    MartyMcFly.module_info()
    (mimic 1.7.4) lib/mimic/server.ex:325: Mimic.Server.handle_call/3
    (stdlib 5.0.1) gen_server.erl:1113: :gen_server.try_handle_call/4
    (stdlib 5.0.1) gen_server.erl:1142: :gen_server.handle_msg/6
    (stdlib 5.0.1) proc_lib.erl:241: :proc_lib.init_p_do_apply/3
Last message (from #PID<0.200.0>): {:stub_with, Calculator, MartyMcFly, #PID<0.200.0>}
State: %Mimic.Server.State{verify_on_exit: MapSet.new([]), mode: :private, global_pid: nil, stubs: %{}, expectations: %{}, modules_beam: %{}, modules_to_be_copied: MapSet.new([NoStubs, Counter, Calculator]), reset_tasks: %{}}
Client #PID<0.200.0> is alive

    (stdlib 5.0.1) gen.erl:259: :gen.do_call/4
    (elixir 1.15.0) lib/gen_server.ex:1071: GenServer.call/3
    (mimic 1.7.4) lib/mimic.ex:186: Mimic.stub_with/2
    (ex_unit 1.15.0) lib/ex_unit/assertions.ex:785: ExUnit.Assertions.assert_raise/2
    test/mimic_test.exs:134: Mimic.Test."test stub_with/1 with non-existant module it fails"/1
    (ex_unit 1.15.0) lib/ex_unit/runner.ex:463: ExUnit.Runner.exec_test/2
    (stdlib 5.0.1) timer.erl:270: :timer.tc/2
    (ex_unit 1.15.0) lib/ex_unit/runner.ex:385: anonymous fn/5 in ExUnit.Runner.spawn_test_monitor/4

I tried in both :global and :private mode.

[apreifsteck]

Thanks for attempting to reproduce it @jimsynz. To confirm, I (belatedly, apologies on that) wrote my own, small reproducer. I saw it is exactly as you described. Even further, I saw the module_info/0 is undefined error occur for only those tests that call stub_with/2 with a bad second module.

The problem I'm observing might be a result of scale. The application I'm working on has over 3000 tests and copies some 80 modules. At this scale I can reproducibly see the behavior I originally described. I can also toggle it by merely changing the name of the second argument in a single stub_with call from something that does exist to something that doesn't. I also saw the exit message you posted present in tests that do not use stub_with and do not at all interact with the misnamed module.

I understand it might be difficult or time consuming to replicate or fix the issue. It does frustrate my I can't just link you all to a github repo and say "here's the branch, if you run the tests...", but alas.

Maybe nothing needs to be done and this issue just hangs about in the archive as a warning to posterity. I probably wouldn't have had this problem if I had been running my full suite more frequently.

[edgurgel]

Yeah I guess the problem is that GenServer Mimic.Server terminating is happening and other unrelated tests will fail instead of just the test that that used stub_with/2 with a module that doesn't exist.