Usage with acceptance/feature tests

[PragTob]

:wave:

Hello again wonderful people making lives of developers everywhere nicer :green_heart:

So, basically turns out stub_with/2 sadly wasn't the solution to my problem as described in #76 (fixed my doc mistake in #80 ), as #41 points out mox is designed to be explicit per process just calling stub_with/2 in test_helper.exs with the intention of setting a global fallback doesn't work.

This is a challenge when wanting to work with acceptance/feature testing libraries like wallaby/hound (and still execute tests in parallel), as we can't set the explicit allowances.

I see 2 options to solve this:

0. something easy that I've overlooked, you'll tell me and I think "WHY DIDN'T I SEE THIS"
1. reopen #76, new API suggestion defmock(mock_name, for: behaviour, fallback_to: implementation) (or stub_with: impl ?)
2. do something similar to the SQL.Sandbox plug in phoenix_ecto

While 2. is arguably cleaner/sticking more with the general concept of explicit allowances it requires a bigger integration (most likely wallaby/hound would need to be made aware of the new metadata they are passed unless there's already a good option to pass these to them) + requires writing the plug etc.

Hence, my favorite would be 1 for now as it seems easier to do and somewhat generally useful, but I'm not a maintainer and I've never done something like 2 so it might be easier than I believe ;) Also having a "global fallback" is an ok option imo as opposed to SQL Sandbox where just running global defeats the purpose.

Thanks, looking forward to your feedback and naturally happy to help with the implementation! :green_heart:

edit: one thing I forgot, with 1 it's impossible to add expectation to the running web server process while with 2 it should be possible. It's not a functionality I want so it might still be worth the trade off.

[josevalim]

I agree with everything. Can we please give 1 a try? Can you please send a PR for us to take a look at it?

[PragTob]

@josevalim thanks! Will do, if my time frame works out I hope to be able to PR it by next week but let's see how this week goes :)

Also I realized something while falling asleep (edited into issue description):

one thing I forgot, with 1 it's impossible to add expectation to the running web server process while with 2 it should be possible. It's not a functionality I want so it might still be worth the trade off

[josevalim]

Yeah, it is fine. We can always build 2 later on top of 1, so I am fine with going with the simplest that works for now.

[hubertlepicki]

We had the idea of 2) when I initially implemented the mode switching to Mox, but I never got to the point of actually implementing the required Plug. The "good enough" solution for me and probably most people was to switch to async: false and use global mode in the Wallaby tests, especially that once you have long running processes in the system you probably switched to async: false for the integration tests already... But I think we still want 2) at some point.

@PragTob maybe it doesn't work for you for some reason when solving the issue described in #76, but you know that "all problems can be solved by adding additional layer of abstraction" ;) may be exactly something you need here.

So you probably already are using Application.get_env at compilation to point your code to use mocks or real implementation code. This, again, is used to point compiler to which module to use at compile time.

How about you add, in test mode only, one layer of abstraction, which would then use application configuration to dynamically dispatch the calls to either mock, or original implementation, depending on application environment setting. Then, in setup blocks of your unit tests, you'd switch to mocks. For integration tests, in setup blocks, you'd switch to implementation.

[PragTob]

@hubertlepicki thanks for the input! That'd require async: false though or am I missing something? I'm not willing to go async: false :) I looked at the code and I think I know where and how to implement it - it's only the question of when to do it :D Plus, as long as plataformatec wants to have it I'm much happier investing more time but solving it for everyone instead of for just one project.

[josevalim]

Ping!

[PragTob]

@josevalim thanks for the ping :green_heart:

Other things turned up on the project and I didn't have personal time to do it yet. Still, I wanna do it. At the latest in December when I'm off of projects - hopefully before :)

Tobi

[SamuelWillis]

I just found this and I'm bumping into the exact same issue.

In our unit tests we are using a Mock API but in our Wallaby feature tests we would like to use our actual API module.

Has there been any progress made here?

My hope was that I could do a Mox.stub_with in the FeatureCase and Mox would simply fallback to the actual implementation in the FeatureCase tests, however I get the following error: no expectation defined for __MODULE_NAME__ when I run the tests.

I'm not sure I fully followed the suggestions in the thread above, but as far as I understood it would require running in async: false which wouldn't be great.

Is there anyway I can help with this?

[josevalim]

Step 1 was implemented but if you need concurrent acceptance tests, someone needs to implement step 2. It is outside the scope of Mox though.

[SamuelWillis]

Step 1 was implemented but if you need concurrent acceptance tests, someone needs to implement step 2. It is outside the scope of Mox though.

Ah great, I hadn't realized step one was done. Thank you!

[josevalim]

Closing as step 1 is done and step 2 is outside of Mox. :) If you need step 2, look at the Plug Ecto Sandbox for inspiration. It is mostly a header with information of a mock to allow for that particular request.