dashbitco / mox

Mocks and explicit contracts in Elixir
1.35k stars 77 forks source link

Module configured with Mox cannot be integration tested #65

Closed LLay closed 5 years ago

LLay commented 5 years ago

If my understanding of Mox is correct, I think I have found a limitation with the suggested configuration. I have a solution to that limitation, but the solution itself introduces other limitations. I'm looking for a solution (or combination of solutions) that will satisfy 3 criteria:

Be able to create a mock for a function such that: a) you can use both unit and integration tests on that function b) the function does not have to be called by your code c) the mock can be used at and arbitrary call depth from the function under tests (ie. the function under test can be called, step down several layers of sub-function calls, and then encounter a mock

The method described in the hexdocs (and original article) for configuring modules to be mocked has the limitation that the module now must always be mocked. Because we are substituting modules at compile-time based on application config it is not possible to only sometimes mock that module. This means that you cannot run an integration test over a module that has been configured in this way.

To get around this, for the function under test, you can give the modules it calls internally as parameters. You can default that parameter to the real module, and pass in the mock module in tests. This solves our "I can't run integration tests" problem. But this approach is limited in two ways: 1) You cannot use it on for functions that you do not define, namely 3rd party library callbacks. Therefore, to test functions on the boundary of your codebase, you must either use the first method (application config) and only ever write unit tests for that function. Or don't create mocks at all, and only use integration tests. 2) Also note that with this strategy, it is difficult to create a mock for a function that is not directly called by the function under test, as you would have to pass a module down through multiple function calls until you got to the function you wanted to create a mock for.

Is there a way that you can create a mock for a function such that the 3 criteria listed above are met?

josevalim commented 5 years ago

You don’t need to define the attribute at compile time. You can also read it at runtime and it will still be quite fast, as it uses ETS.

You can also use stub_with, to have a mock behave like an existing module. --

José Valim www.plataformatec.com.br Skype: jv.ptec Founder and Director of R&D

LLay commented 5 years ago

Thanks José!

josevalim commented 5 years ago

Closing this, please let us know if you have further questions, thanks!