Protractor Tests without a Backend (or how to mock a backend)

[mren]

Hi!

I want to write integration tests for an angular app with protractor. As @juliemr mentioned in a comment to a similar issue written by @mrappleton, Protractor is intended for End to End Tests with an actual backend hooked up. This is a reasonable use case and good to know.

However in my case I have a fairly well tested Backend. Therefore I would prefer to have my protractor tests not depend on my backend.

The advantages of this could be

• Testing the angularjs app can be done in separation of the server
• I won't test things twice
• Less dependencies to worry about

I played around with something similar to @leifhanack backend dsl. It is possible to test the whole App and Mock the Api Routes in the Test. However this feels like a lot of overhead (setup rest server in each test, worry about body parsers,...)

In my App I have a Api Module that talks to the server. I would really like to spy and mock this Api Module from the tests. This is difficult as the tests (protractor) are not executed in the same scope as the app (selenium driven browser).

I would fancy to do something like this in my tests:

Api = $injector.get('Api');
sinon.mock(Api, 'getSomethingFromServer').andRespondWith({foo: 'bar'})
// do tests, go to page, fill out form, submit form, etc...
assert(Api.getSomethingFromServer.wasCalledOnce);

In this case I could test the client in seperation without doing a lot of boilerplate to get up running. I know that this is not possible right now, because $injector is running in the browsers scope.

The only way to do something similar would be to use ptor.addMockModule. This is limited, as it is only possible to replace whole modules (and not mock parts of it) and it is difficult to make assertions on the mocked functions (something like mockedModule.function.wasCalledOnce)

What would be a great way to mock the access to the backend to test an angular app in isolation?

What is the angularjs teams view on this issue?

[wakandan]

I think I'm doing something similar to what you're wanting to setup, don't remember where I read about it but here are the steps:

I have a mocked backend module mocked-backend.js with

exports.httpBackendMock = function() {
    angular.module('httpBackendMock', ['mainApp', 'ngMockE2E'])
    .run(function($httpBackend) {
        console.log('Test platform bootstrapping');  
        ...     
        $httpBackend.whenGET('/events').respond([sampleEvent]);        
        $httpBackend.whenGET('/events/' + sampleEventId).respond(sampleEvent);
        $httpBackend.whenGET('/login').passThrough();
        $httpBackend.whenGET(/partials\/.*/).passThrough();        
        $httpBackend.whenPOST('/events').respond(function(method, url, data) {
            data._id = 123456789;
            return [200, angular.toJson(data), {}];
        });
        $httpBackend.whenDELETE('/events/' + sampleEventId).respond(function(method, url, data) {
            return [200, {
                delete: sampleEvent
            }, {}];
        });
        console.log('Test platform bootstrapping ... done');
    });
}

where the mainApp is name of the real apps, in which all the routes are declared. Notice the dependency to ngMockE2E, you will need to have reference to angular-mocks.js in your index.html file for this to work.

In my protractor scenario file:

var mockModule = require('./mocked-backend');

And in your test in your beforeEach block you could put

beforeEach(function() {
     ptor.addMockModule('httpBackendMock', mockModule.httpBackendMock);        
});

The only I couldn't figure out was setting response header with Set-Cookies so that the cookie part of my app could be tested. I would like to hear some insight about it as well.

[mren]

Hey @wakandan :)

Nice to see more people with the same problem. You probably read about this in issue 31.

From the look at your code, it seem to me that the mock data (like sampleEvent) have to be defined in the scope of exports.httpBackendMock. The code will be stringified and evaluated in the scope of the browser. Thus it is not possible to make this method more abstract that every test can use a suitable version of this mocked version (like a partial with a generic sampleEvent variable). @juliemr pointed this out in the code she wrote to Issue 31. I did something similar to this in a project with phantomjs and it worked in the short term. In the long term the tests became difficult to write (read error prone). Working with two scopes in a test seems to be difficult.

The way you approached it is as far as I can see the only feasible way to approach mocking angularjs apps from protractor. In this approach a complete Module is replaced with something different and is put into a different scope which is not easily accessible from the test.

This has these three disadvantages:

• Each Mock will replace the whole module, thus each method that will be used has to be mocked, too. Even if it is not neccessary
• Each Mocked Module has custom hard coded data. This gets difficult if you have several ways to return data (Return value, Callback, Promise)
• No spying on only needed methods. This would touch the relevant data only and could return more information, like the recorded calls of the spys to test more assertions (spyedApi.getUser.withArgs(foo, bar).CalledOnce)

However I'm not aware of any better way! I'd love to hear the opinions of people involved in this subject in any way. Either as a angularjs developer with better insights into the structure of angularjs and protractor or as somebody who has written integration tests with protractor that solve these issues.

Best, Mark

[wakandan]

Please correct me if I understand your points correctly...I'm quite a newbie in this area and the app I'm developing is still in early stage so I should not have enough experience yet :) I'm just trying to understand what I might hit when my application gets bigger in the future...

In this approach a complete Module is replaced with something different and is put into a different scope which is not easily accessible from the test.

The act of separating the mock backend into a separate module is just for code organizing purpose in my case...Originally I was putting them inside an beforeEach block in my test, so should it be possible to alter the mocked backend depends on the test's need...?

Each Mock will replace the whole module, thus each method that will be used has to be mocked, too. Even if it is not neccessary

I don't quite understand this point... I am yet to be sure how the mocking is actually being done in protractor code, but the way it's documented to be defined looks a lot like that in the old way of e2e test at http://docs.angularjs.org/api/ngMockE2E.$httpBackend. The mocked module replaces the main app in the bootstrap process and mocks $httpBackend to achieve the effect of mocked requests, but the rest of the app doesn't need to be mocked. It wasn't possible to inject the mocked backend directly like in protractor now....

[lucassus]

First of all the goal of integration testing is to test everything. Therefore mocking the backend in this case isn't a good idea. From my perspective the real problem with testing in protractor (and with any other tool for integration/browser testing) is how to load database fixtures or how to setup the initial state of the app. In ruby world we have excellent capybara gem which can directly hook into the database but in the javascript world things are much more complicated. Especially when the backend is written in the other language, framework, etc.

In my sample project (https://github.com/lucassus/angular-seed) I workaround this problem with dummy API endpoint for loading db fixtures (obviously it should be exposed only in the test environment). Inside the integration test I'm using this endpoint to load fixtures in order to ensure that each test has the same initial state.

Here's my API for loading backend fixtures https://github.com/lucassus/angular-seed/blob/baf9b8aa110aa8b68fdd4a52afd6b8f9dba52adc/server/lib/app.coffee#L45 And how I use it along with protractor: https://github.com/lucassus/angular-seed/blob/baf9b8aa110aa8b68fdd4a52afd6b8f9dba52adc/test/protractor/helpers/fixtures.coffee https://github.com/lucassus/angular-seed/blob/baf9b8aa110aa8b68fdd4a52afd6b8f9dba52adc/test/protractor/products_scenario.coffee#L15

The other solution might be a simple proxy server which can intercept all API calls to the backend, record them and replay when the test is run again. Here's my proof of concept: https://github.com/lucassus/vcr-proxy The idea is based on ruby vcr gem https://github.com/vcr/vcr It works for one of my small apps but I'm aware that it might be very painful in large apps.

[vavd]

@lucassus, what do you think about managing the responds from the test?

For example it needs to test a main page. From my poin of view the code in the test should look like:

1. point browser to url
2. wait request
3. push respond
4. assert the changes on the page. Of course the changes depend on the respond

The test will fail if request doesn't come or the page is changed wrongly.

It seems this is a good way of testing a single page app.

[seglo]

Just throwing my 2 cents in here.

I also have the desire to mock out the backend. I understand the need for some to write integration tests that include the backend API, but it would be awesome to be able to develop the front end independent of the constraints of the backend (and any other dependencies it may also have). From what I've read about angular-mocks and the original e2e solution, it would have been a simple solution to mock out a few of your backend calls, run the karma server and you're done. This would be my ideal scenario, but given the messaging on the e2e documentation in angular's docs that it won't be supported for much longer I don't want to invest too much time in that solution.

Some of the solutions here are proposing the use of a proxy server to do the same thing, but this just seems like one more thing to setup when running e2e "angular app" tests. I already have the selenium server requirement, now I need to run a proxy/shim backend too?

Am I missing something obvious? Don't other developers have the desire to develop the front end independent from their backend?

@lucassus All that said, I still am interested in learning more about vcr-proxy. I'm going to spend some time looking at it now.

[lucassus]

@vavd when it comes to managing the responds from the test. I see two options:

• if we can easily load test fixtures on the backend/API side we should do it. As I said the goal of integration testing is to test everything
• mocking http request inside the specs might a good solution when backend setup is very complicated or the app takes ages to boot, etc.

[lucassus]

Some of the solutions here are proposing the use of a proxy server to do the same thing, but this just seems like one more thing to setup when running e2e "angular app" tests. I already have the selenium server requirement, now I need to run a proxy/shim backend too?

@seglo My vcr-proxy solution is very lightweight and it takes literary two seconds to run. Always you can create a simple bash script to kick of the proxy and run protractor (for example https://github.com/lucassus/angular-seed/blob/b35918e4194ef455c4bb1743b371b523e3c6342b/script/test-integration) or wrap everything in grunt task.

[lucassus]

@seglo you could also check https://github.com/vcr/vcr/issues/187

[tdumitrescu]

In case anyone's interested: I recently ran into similar issues of wanting to control (not mock) an Express backend from within E2E tests, for purposes such as database-cleaning, fixture loading, and shortcutting session/cookie logic, so I made a lightweight middleware package that allows you to inject test-only API endpoints into your server: https://github.com/tdumitrescu/test-requests. The principle is similar to the setup @lucassus described above, but with a generalized interface and code to ensure that the routes are only exposed in the test environment. If you're looking to do real integration testing with a node.js backend, you might find it helpful.

[jonbcampos]

If anyone is still watching this thread I would very much appreciate help with accessing angular from a protractor test. I think it is important to be able to adjust your data for even some of the most basic tests that I would expect this product to handle.

I'm agreeing with some of the other posters that think this is important.

[wilkerlucio]

Well, here I'm trying to work with the addMockModules to mock my services stuff (which in my case is actually Firebase, so, I have an external backend, which makes very problematic to handle the DB cleaning/fixtures).

Here is an example of what I'm doing:

        browser.addMockModule('myapp.data-points', function() {
            angular.module('myapp.data-points', [])
                .factory('databaseRef', function() {
                    return function() {
                        return [];
                    };
                });
        });

In my case, I decided to make a wrapper into Firebase connections, and using the path as a generic value (so I can migrate from Firebase in future if I want to). So, all my connections are basic generated from a path using this databaseRef factory, so for all external data I only need to mock this factory and I'm basically good.

Thinking ahead I would like to "simulate" my db using this as bases, but that evolves a lot of data, and since the contexts of the addMockModule function is not same (I can't use scoped vars) it gets very tricky... The problem is that addMockModule doesn't accept arguments to be sent to the context (they could be, since executeScript can).

I'll try to hack protractor a bit to enable arguments to be sent into addMockModule.

Please let me know what you guys think on this approach.

[unDemian]

Hey guys, I created a little customizable mock module to help me handle success and error scenarios, maybe it will help you better organize mocking.

https://github.com/unDemian/protractor-mock

[kbaltrinic]

@mren. I was looking to do the exact same thing as you--use Projector to just test my app--and came across this issue when was looking for a solution. Not finding a good one, I have created an $httpBackend proxy that uses browser.executeJavascript calls to allow one to configure the real $httpBackend from within your Projector tests. Have a look.

https://github.com/kbaltrinic/http-backend-proxy

[RaviH]

I would love the flexibility to mock out response from the server (REST API) via httpBackend like in Unit Test. Mind you I am a newbie to angular, but from what I have read so far, using httpBackend is cumbersome to setup and you cannot change your response for each spec (at least not without jumping through many hoops).

+1 on testing UI with mocked responses from the server.

[fiznool]

If anybody is willing to use a well-tested external library, you can include and use SinonJS to mock backends. It works by hijacking the XHR object and responding with an object you define. The SinonJS fakeserver can be used standalone in case you don't want the whole Sinon library (however, the library as a whole is very useful too).

[skoczen]

+1 to https://github.com/kbaltrinic/http-backend-proxy - works beautifully, no hacks to my app necessary. Thanks, @kbaltrinic !

[seglo]

I forgot about this post. I actually just wrote a connect middleware that satisfies my needs. Works with grunt or gulp.

https://github.com/seglo/grunt-connect-prism https://github.com/seglo/connect-prism

[spra85]

@kbaltrinic or @skoczen have you been able to get the http-backend-proxy also using partial templates? I've tried letting those pass through the proxy using passThrough() but having problems getting the template to display. Curious if there was a working example somewhere of that

[kood1]

If anyone's interested, please check out MSL project - https://github.com/FINRAOS/MSL. We released this last month and it's used for mocking back-end for locally running app (client-side code only). We originally used ng-scenario, ng-mockE2E for testing Angular apps (loved it) but we wrote a generic tool which can be used for testing any apps (jQuery, etc). We currently use MSL with Protractor, Selenium, straight Jasmine/Mocha tests. There are JS and Java bindings available.

Check out the intro video here - http://finraos.github.io/MSL/gettingstarted.html

[nadersoliman]

This is a promising and fresh solution https://github.com/atecarlos/protractor-http-mock

using ngMockE2E is too intrusive the above solution doesn't depend on ngMockE2E altogether, it is still rough of the edges but works.

[atecarlos]

Thanks nadersoliman! I'm very happy you found the plugin useful. I'm open to any suggestions or comments on how to make the plugin better.

[mcalthrop]

Are there any plans to update the documentation to include anything about mocking service responses?

From what I can see (although very well could have missed it), there is no such information at the moment – to me, it seems to be a feature that is absolutely necessary when developing an enterprise-level application.

[drewstewart]

You could mock your backend with Sandbox - configure your angular app to point at it when running tests.

[shoshanatech]

I can't get any httpBackend mock working with my seemingly simple login service. I've tried both atecarlos and kbaltrinic's libraries - each does exactly the same - if I start the real server, the test works fine. If I turn off the server and run only with mocks, they fail, leaving me believing that httpBackend is not being mocked. Here's my test:

var HttpBackend = require('http-backend-proxy');
var proxy = new HttpBackend(browser);
var loginUser;  // fixture with some users

describe('login', function () {
beforeEach(function () {
    browser.get('http://localhost:8100');  // get root
    browser.executeScript("localStorage.removeItem('activeUser');");  'make sure token is empty'
    browser.sleep(2000);
    loginUser = require('../login.json');
});

beforeEach(function () {
    browser.ignoreSynchronization = false;
    // regex for http://(myserver)/authenticate/login?username="aname"
    proxy.whenGET(/\/login\//).respond(loginUser[0]);
});

afterEach(function() {
    browser.manage().logs().get('browser').then(function(browserLog){
        if(browserLog.length) console.log('log:'+ JSON.stringify(browserLog));
    })
});

it('should go to mainmenu after successful login', function () {
    element(by.model('username')).sendKeys('tlouisa');
    browser.sleep(2000);
    element(by.id('btnLogin')).click();
    browser.sleep(2000);
    expect(browser.getCurrentUrl()).toEqual('http://localhost:8100/#/mainmenu');
});

[atecarlos]

Hi @shoshanatech. Feel free to submit an issue on protractor-http-mock so we can work out whatever issue it is you are having.

[kbaltrinic]

@shoshanatech I'll echo @atecarlos sentiments--whichever library you prefer to use, post an issue to the appropriate one and we'll do what we can to help. You may also want to take a look a the closed issues. I can't speak for protractor-http-mock, but most of the common issues for the http-backend-proxy have come up and there is at least one closed issue dealing with it.

Without knowing more details, at a guess, I would say that you are not loading the ngMockE2E module in the browser. Protractor-http-mock does not depend on this module, but the http-backend-proxy does.

[rfodge]

Just wondering if anyone has a good solution for updating mocked called within a protractor test. I need to be able to mock a call (which I can do using ngMockE2E) but the next time the call is made, I want it to come back with different results.

httpbackend.when('GET', ....URL....).respond(200, results); where results is a json object that is returned

The first time the call is made is it coming back with the correct json. But within the same test, I want to update those results so that the next time the call is made, it returns the updated json.

Thoughts?

[kbaltrinic]

@rfodge, The solution to having the same call return different responses is to place logic in the response call back function to return different responses under different circumstances.

If you are using the http-backend-proxy, you can use the context object to accomplish this. Simply have the callback function return a value defined on the context object, then change the value and manually re-sync the context between tests. Here are the relevant docs.

[rfodge]

Thank you! I was able to get that working based on the responses on the issue thread in github. :)

[nabil-boag]

A project called Angular Multimocks solves a lot of the problems people are having here. It is a wrapper around $httpBackend that allows you to create mock files as simple JSON files and them compose mocks into various scenarios.

[germanio]

Hope this is still active. I have a question about mocking the backend:

How do you organize the mocks when the app grows?

I mean, if we start mocking $httpBackend calls for each test, we quickly end up having a big list of .whenGET(/foo\/bar/).respond(mockResponse); to maintain in a single file, and also duplicated for each test, right? We managed to add the data for the response (the one that goes in mockResponse) from a different file, which makes much easier to maintain and read, but after a while, those files are again big enough to make maintenance a problem. So I tried to divide that list in at least two calls to browser.addMockModule() that adds different module.run(), but the last one added seems to overwrite the first one. I don't understand why that happens since each call adds a different module, and each module.run() is supposed to add mock calls to the same $httpBackend (mock) instance, right? Any thought on how to DRY my mocks will be appreciated. PS: I will take a look at what @nabil-boag has shared, it looks promising.

[nabil-boag]

Hey @germanio, Angular Multimocks does help solve the problem you talk about by allowing you to define mocks once and then load them up in different scenarios.

Full disclosure, I am the maintainer of Multimocks 😸. If you need any help getting set up let me know.

[jonbcampos]

All that are watching this. I haven't worked with Multimocks, though I am sure it is a great solution. In my last project I set up a mini framework around protractor http mocks with great success. There was a bit of setup (hence the mini framework) but it did work well for me.

Ultimately protractor tests are e2e tests and therefore should be hitting real services. I'm sure we all agree on that already though.

[nabil-boag]

Hey @jonbcampos,

If your goal in testing is to do a full integration test of your stack then mocking the backend is probably counter-productive.

I think a lot of people want to use protractor to acceptance/service test their front end applications in isolation of a back end. My argument though is that real backends are inconstant and inflexible. If you use a real backend, ask yourself the following questions:

• What happens when my test fails?
• Was it my application or did the internet connection fail?
• Was the API team deploying code when I was running my test?
• What happens if the API is offline or has a version deployed that conflicts with my application?

If your application makes very few API calls or the UX doesn't change based on different API responses then Angular's built-in ngMock mocking tools are great.

However, as applications grow in size and complexity, integration tests become increasingly flakey (non-deterministic). Mocks allow you to control the test process carefully, but it can become difficult to manage API response mocks, change scenarios and test complex UX interactions (which is what people like @germanio are experiencing).

[blabno]

Guys, to you really run all the edge cases with real backend? Mocks are a must!

2016-04-13 11:27 GMT+02:00 Nabil Boag notifications@github.com:

Hey @jonbcampos https://github.com/jonbcampos,

If your goal in testing is to do a full integration test of your stack then mocking the backend is probably counter-productive.

I think a lot of people want to use protractor to acceptance/service test their front end applications in isolation of a back end. My argument though is that real backends are inconstant and inflexible. If you use a real backend, ask yourself the following questions:

• What happens when my test fails?
• Was it my application or did the internet connection fail?
• Was the API team deploying code when I was running my test?
• What happens if the API is offline or has a version deployed that conflicts with my application?

If your application makes very few API calls or the UX doesn't change based on different API responses then Angular's built-in ngMock mocking tools are great.

However, as applications grow in size and complexity, integration tests become increasingly flakey (non-deterministic). Mocks allow you to control the test process carefully, but it can become difficult to manage API response mocks, change scenarios and test complex UX interactions (which is what people like @germanio https://github.com/germanio are experiencing).

— You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub https://github.com/angular/protractor/issues/125#issuecomment-209332232

[jonbcampos]

While I always appreciate a long over explanation, I already agreed and that is why I've said I use protractor HTTP mocks with great success. On Apr 13, 2016 6:48 AM, "Bernard Labno" notifications@github.com wrote:

Guys, to you really run all the edge cases with real backend? Mocks are a must!

2016-04-13 11:27 GMT+02:00 Nabil Boag notifications@github.com:

Hey @jonbcampos https://github.com/jonbcampos,

If your goal in testing is to do a full integration test of your stack then mocking the backend is probably counter-productive.

I think a lot of people want to use protractor to acceptance/service test their front end applications in isolation of a back end. My argument though is that real backends are inconstant and inflexible. If you use a real backend, ask yourself the following questions:

• What happens when my test fails?
• Was it my application or did the internet connection fail?
• Was the API team deploying code when I was running my test?
• What happens if the API is offline or has a version deployed that conflicts with my application?

If your application makes very few API calls or the UX doesn't change based on different API responses then Angular's built-in ngMock mocking tools are great.

However, as applications grow in size and complexity, integration tests become increasingly flakey (non-deterministic). Mocks allow you to control the test process carefully, but it can become difficult to manage API response mocks, change scenarios and test complex UX interactions (which is what people like @germanio https://github.com/germanio are experiencing).

— You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub <https://github.com/angular/protractor/issues/125#issuecomment-209332232

— You are receiving this because you were mentioned. Reply to this email directly or view it on GitHub https://github.com/angular/protractor/issues/125#issuecomment-209387995

[nabil-boag]

Sorry for the long comment @jonbcampos.

[jonbcampos]

no worries. I was more hoping to stem the tide of others that were starting to jump on board thinking we weren't in agreement.

On Wed, Apr 13, 2016 at 9:22 AM, Nabil Boag notifications@github.com wrote:

Sorry for the long comment @jonbcampos https://github.com/jonbcampos.

— You are receiving this because you were mentioned. Reply to this email directly or view it on GitHub https://github.com/angular/protractor/issues/125#issuecomment-209471696

Jonathan Campos

[juliemr]

I'm glad there's been good discussion here, and I think people have brought up good points about the advantages and disadvantages of mocking out backends. I think these are good questions to ask, but I'm not sure that this issue is the best place to do them. We're cleaning up issues, so I'm going to go ahead and close this one.

[jdnichollsc]

Thanks for the examples! Guys, check my example without more npm modules, you can clone and test! yay! 💃 https://github.com/jdnichollsc/Game-of-Drones/tree/master/FrontEnd/tests/e2e-tests

[evtk]

I have ngMockE2e in place to return mocks as separate json file, based on a synchronous XMLHttpRequest. I have created a decorator for the ngMocke2e httpbackend to be able to manipulate the response time of the various backend calls that don't go with a PassThrough.

$provide.decorator('$httpBackend', function ($delegate) {
var proxy = function (method, url, data, callback, headers) {
  var waitTimes = [
    { regex: /^.*BackendCallUri1.*$/, response: 3000 },
    { regex: /^.*BackendCallUri2.*$/, response: 1500 }
  ]; 
var responseTime = 0;
var found = false;
  angular.forEach(waitTimes, function (waitTime) {
    if (!found && waitTime.regex.test(url)) {
      responseTime = waitTime.response;
      found = true;
    }
  });
  var interceptor = function () {
    var _this = this,
      _arguments = arguments;

    setTimeout(function () {
      callback.apply(_this, _arguments);
    }, responseTime);
  };

  return $delegate.call(this, method, url, data, interceptor, headers);
};
for (var key in $delegate) {
  proxy[key] = $delegate[key];
}
return proxy;        

As you can see the default responsetime is 0, but you can configure each call separatly. Anyways.. the thing is that protractor doesn't seems to catch these waittimes. If I set them all to 0, test runs ok. But as soon as I configure a wait time, protractor doesn't wait for the response to be returned. Failing my tests. Is what I have achieved with the wait times not supported by waitForAngular() ?

BTW: I'm still on angular. 1.5.8. Upgrading to 1.6.2 for this fix didn't solve the issue.

Any suggestions?

[Pragati103]

Hi it is possible to use mockservice in e2e testing

[Pragati103]

Any suggestions?