Protractor fails when using aggressive delete

[dmitrybelyakov]

Hi, It caused me some frustration until I finally traced it down. So let me explain:

When writing end-to-end tests using angular's new protractor webdriver the tests fail when you use aggressive delete option. The reason for that is that you use $timeout service to remove caches after timeout as in here.

The way protractor works is that it waits for angular-known services like $http, $q, $timeout and others to complete/resolve before executing next test statement to allow you not to set explicit timeouts in your tests. In other words it will wait for angular to complete before proceeding.

This causes protractor tests to fail with a timeout whenever you put to cache because it waits for your maxAge timeout to resume testing.

[jmdobry]

As far as I understand it, $timeout isn't supposed to work the same way in a testing environment. You have to tell it when to resolve/reject promises.

Try injecting $timeout into your tests and doing $timeout.flush() to manually flush the promise queue of any pending promises. I have to do this in the tests for angular-cache because in the testing environment $timeout has a mock implementation.

See https://github.com/jmdobry/angular-cache/blob/master/test/angular-cacheSpec.js for how I handle $timeout in tests.

See how this works out for you...

[dmitrybelyakov]

I'm not sure if that is at all possible because you test against a protractor-driven browser how the end-user sees it. So I'm not sure if you can actually inject anything...

[jmdobry]

How long are you setting maxAge to? Shouldn't Protractor resume testing as soon as the $timeouts finish?

[dmitrybelyakov]

It's set to 15 minutes, yet protractor sets timeout for each one test to 5 seconds so that's no surprise. If I set maxTimeout to less than that everything works fine.

Additionally I found a bit hackish way to get the injector by executing inline script with webdriver like so:

ptor.executeScript('angular.element(document.body).injector().get("$timeout");');

But unfortunately it has no flush() as it is only available in mock $timeout

[jmdobry]

I can see why 15 minutes is not a practical amount of time to wait! Seems a little strange that Protractor waits synchronously for something that by its nature is asynchronous.

[dmitrybelyakov]

I think the rationale behind this was that it's better to wait then to have fragile explicit pauses each other line in your tests. Perhaps there is (or will be) a workaround for that. I guess it's better to ask @juliemr directly about that.

Alternatively it is possible to use your own timeout service that protractor is unaware of.

[dmitrybelyakov]

The similar issue is discussed in here angular/protractor/issues/49

[juliemr]

Hi! Yup, the main idea of waiting for $timeouts to finish is that flaky tests are useless and waste everyone's time, so we err on the side of spending longer to make sure everything is repeatable. I'm brainstorming ways of telling protractor that there's an expected $timeout, but for the moment, changing maxAge or using ptor.ignoreSynchronization is the way to go.

Alternatively, depending on your situation, you can mock out a service for a page Protractor is testing using ptor.addMockModule (https://github.com/angular/protractor/blob/master/lib/protractor.js#L529). You could inject something to overwrite $timeout, though that might have other bad consequences.

[jmdobry]

@juliemr I agree.

When using $timeout in any testing situation it is probably best to keep the delay as short as possible.

aggressiveDelete will continue to use $timeout, but this feature may make it easier to work with protractor tests in the future, since it won't use $timeout.

[jmdobry]

Closing this because it isn't a problem to be fixed in angular-cache–protractor eventually times out when long $timeouts are used because protractor synchronously waits for them.