search

jjh42 / mock

Mocking library for Elixir language
MIT License
656 stars 81 forks source link

Tests sometimes fail with `** (MatchError) no match of right hand side value: false` #111

Open Victoria91 opened 5 years ago

Victoria91 commented 5 years ago

In some our tests we have plenty of mocks and sometimes tests fail with 

     ** (MatchError) no match of right hand side value: false
     code: with_mocks(mocks) do
     stacktrace:
       test/test.exs:267:  <line with `with_mock` call>
       (elixir) lib/enum.ex:1940: Enum."-reduce/3-lists^foldl/2-0-"/3

Everything is defenitely Ok with mocks itself - tests usually pass, but still it is rather annoying when you have lots of such tests and they sometimes fail for no reason :(

Olshansk commented 5 years ago

So is the same test failing/passing ephemerally?

Could you provide some example code so we can debug together?

Victoria91 commented 5 years ago

Here is shortened version of test that still reproduces the error, I hope It will be engouh.

Here is is sample of failed test output

  1) test #init assert 6 (MyAppRouter.ConsumerListenerTest)
     test/consumer_listener_test.exs:101
     ** (MatchError) no match of right hand side value: false
     code: with_mocks(mocks) do
     stacktrace:
       test/consumer_listener_test.exs:104: anonymous fn/2 in MyAppRouter.ConsumerListenerTest."test #init assert 6"/1
       (elixir) lib/enum.ex:1940: Enum."-reduce/3-lists^foldl/2-0-"/3
       test/consumer_listener_test.exs:104: (test)

And one more detail: Our project uses umbrella. The case with flicker failing tests reporoduces only when running tests from the root direcrory of the whole project. When running from the directory of the application with specific test - the test always pass

Olshansk commented 5 years ago

Hmm...

Given that you said this: When running from the directory of the application with specific test - the test always pass and looking at the test you pasted, the only theory I have is that your tests are asynchronous, so there's a weird race condition happening when you are trying to mock things.

Can you try updating this line with use ExUnit.Case, async: true and see if the error persists?

Reference: https://hexdocs.pm/ex_unit/ExUnit.Case.html

Victoria91 commented 4 years ago

No, tests are not asynchronous, if add async: true, that way they will be asynchronous and they won't pass at all but with other errors. Sorry for late reply