Provide a way to write unit tests for the view (and the DOM)

[sylvain-hamel]

We need a way to unit test views and assert the state of their DOM properties.

For instance I might want a tests like this:

describe('a custom element that has an input bound to one of its model property', function() {

  let customElement;
  let domElement;

  beforeEach(function() {
    customElement = aurelia.some_way_to_compile('<my-custom-element></my-custom-element>');
    customElement.name = "joe";

    domElement = aurelia.some_way_to_get_DOM_element(customElement);
  });

  it('should have the input set to the model value', function() {
    expect($(domElement).find('input').val()).toEqual("joe");
  });
});

I posted this question on SO and @EisenbergEffect told me that it's not supported yet and that BehaviorInstance.createForUnitTest is made to test the controller and the model, but not the view.

This plunkr shows how in the Angular world I can use the $compiler service to do that type of test (as well as how to mock directives).

We need the same thing for Aurelia.

PS: This is not about end-to-end protractor tests.

[zewa666]

Could you clarify what the benefit of this would be? What Angular was promoting with $compile was to perform some mix of unit / end2end test in order to verify directives. Yet this clearly isn't really a best practice and resulted because of lack of API support.

In Aurelia you'd typically test the ViewModel not the View. Same applies not only for Pages but also Custom Elements. Your test above just verifies that Aurelias Binding Engine works, which is not something of concern to your test typically, nor does it actually test a real unit of work

[sylvain-hamel]

I agree that you'd typically test the ViewModels (and the Controllers) but I think we need a solution for less typical cases as well.

In my current angular app, 90% of the unit tests are not compiling directives. But the 10% that do are important because they tests key UI behaviors.

Given a view that uses a jQuery plugin (say a charting component). In order to prove that you are configuring the component right, you may have to instantiate it in the DOM and assert its state (expect($(myChartElement).find('.data-point').length.toBe(1)).

I have a few other cases where the UI behavior relies on CSS (collapsible panels). If someone changes the CSS, I want a unit test to fail.

I don't want to have this type of test in my e2e environment for two reasons:

• The above tests have nothing to do with what we tests in our e2e tests (verify login page, verify order approval process, etc…).
• e2e tests tend to have a lot of setup and tend to be slower. They also tend to be executed as an "after push" thing. I want the tests I've described above to be run locally as soon as I change a file.

Moreover, I think a lot of projects don't have e2e tests at all. If Aurelia allows us to do this type of testing in a unit test, then that’s better than not testing at all.

Does that make sense to you?

[plwalters]

@sylvain-hamel my $0.02 here -

1. Testing that the jQuery charting library received the data by stubbing it out is usually sufficient in this case, no? If you test that the charting library is charting the data properly aren't you just testing the tool you are using to see if it works correctly?
2. If you are testing that the CSS wasn't changed why not just put a watch on the file or something? Checking that the CSS you wrote gets applied means you write the same code twice. Once in the CSS file, and once in the test. If you are working on a team I assume that if someone changes the CSS and a test fails that they would simply remove what they removed in the CSS and move on.

That being said if you did want to test how things are rendered in the browser I think they definitely belong in the E2E tests because otherwise I think you'll find that the unit tests wont act 100% like you expect the DOM to when you are deploying and you are going to end up with a bigger headache.

[sylvain-hamel]

@PWKad I understand your points. Both e2e tools and unit tests tools can solve this problem. My preference is to put application logic assertions in e2e tests and technical assertions in unit tests.

[jmwnoble]

My $0.02 having just found the angular $compile really useful in cases where we have written a directive that has quite a few inputs, e.g.

<custom-thingy setting1="vm.value1" setting2="vm.value2" setting3="vm.value3" etc=...></custom-thingy>

We want to test two things 1) that given known values the element itself gets constructed as it should, in terms of its text, css, or other attributes like tooltips and 2) that instances of it in our app are invoked correctly.
In the first case it is testing whether our custom html and model are working together in a controlled setting. This isn't a candidate for an e2e test, it's just extending the code under test to include the html of the element. In the second case we want to enforce that all instances are invoking the element correctly, so that if someone writes:

<custom-thingy setting="vm.value1" satting2="vm.value2" setting3="vm.value3" etc=...></custom-thingy>

it fails because there is no setting and no satting1. An e2e test could do this, but testing this directly is quicker to run and quicker to resolve.

[jmwnoble]

Actually it's more than that - since I've started using Wallaby.js for more TDD-style coding in angular or aurelia, I've wanted to see the html output as I'm coding the custom element. Not something you can do with e2e.

[zewa666]

@sylvain-hamel so essentially, as @PWKad said, the given example is again testing and verifying a foreign system which is not part of the code you're testing. Mocking and stubbing should be the way there.

@jmwnoble your example is similar, testing the rendered result is always a bit closer to E2E than Unit tests. I do agree though, that sometimes it does make sense. E.g. for the CSS-Animator we've used something similar, not a pure Unit test, but neither a real E2E test. It essentially just leverages DOM-Fixtures and then queries elements and their modifications https://github.com/aurelia/animator-css/tree/master/test

So to sum it up this really is about:

• creating a test environment
• starting up Aurelia
• compiling pages / elements
• access DOM

Tbh I'm not sure how much would be involved to do that. As long as you stay in Chrome or another concrete browser as testrunner maybe, but as soon as you go on towards node based or headless browsers it can become quite tricky. Although the PAL abstraction now might help here.

Lets see what @EisenbergEffect thinks about this one

[sylvain-hamel]

@zewa666, I didn't want to go into a lengthy discussion about those specific examples and what can be tested how. It's debatable and is mostly a matter of opinion or context. I think Aurelia should give us the option to do it that way if we I choose to.

Regarding headless browsers, my current AngularJS tests run fine with PhantomJS. I haven't found assertions that don't work in that context yet. I don't know about "pure node"; maybe they'd require something to provide a browser-like environment for the tests to run.

[EisenbergEffect]

I would like to provide something like this, if only on principle, that everything should be unit testable. I don't think we will have it in time for the beta, but we can add it in the future.

[atsu85]

:)

[jdanyow]

Here's a prototype: https://github.com/aurelia/templating-resources/pull/153 The repeat custom attribute has so many moving parts and so much surface area to test that we're adding a suite of integration tests.

[sylvain-hamel]

@jdanyow nice work. It would be great to extract the first 120 lines into an IntegrationTestHelper class.

let integrationTestHelper = new IntegrationTestHelper();
beforeEach(){
   integrationTestHelper.setup(MyCustomeElement1);
   integrationTestHelper.setup(MyCustomeElement2);
}

// add your tests here

afterEach(){
   integrationTestHelper.tearDown(MyCustomeElement)
}

A reusable async assertion queue would be useful too.

Until then, I'm going to try and apply your solution to my case and I'll report feedback here.

[jdanyow]

For sure! We're planning on getting this cleaned up and exposed as an API. Just wanted to share early on so people could start using and providing feedback/questions/use-cases

[EisenbergEffect]

Yep, I think it's important that we put it through its paces. If we can handle all our own testing scenarios with it, then it's likely to work for the community. There are some other cases here that would need to be accounted for with other components. For example, this isn't accounting for view loading, etc. So, we'll probably need to re-work how the components are created for tests. But, this is a good start and we can use it to move forward with testing the repeater, which is pretty high priority.

[EisenbergEffect]

It's not ready yet and we want to probably add some more features, but we have something like this we're working on:

describe('the test component', () => {
  it('renders foo into its view', (done) => {
    ArrangeComponent
      .withResources('src/test')
      .withView('<test foo.bind="firstName"></test>')
      .withViewModel({ firstName: 'Rob' })
      .assert(element => {
        expect(element.innerHTML).toContain('Rob');
        done();
      });
  });
});

[sylvain-hamel]

That's great! Will you consider creating an new aurelia-test-helpers project where you'd group all test related helpers? Earlier in this thread @jdanyow was talking about an async assertion queue, which would be useful. Also, this week I talked with people in Gitter about a helpers to mock httpClient and setup expected requests and responses. All this could go into that projects maybe.

About the above code snippet: In order to be able to pass in a mock, it would be useful to call .withResource() and pass in the instance itself instead of the path to the file/module.

Is this work in progress visible in an unofficial branch/repo somewhere?

Thanks

[EisenbergEffect]

The wip is just in my own local skeleton where I'm writing some tests on components to see how it works. We've been chatting about the api. I think Jeremy has a nice idea to provide more control for testing. Here's his api idea:

describe('the test component', () => {
  let component;
  beforeAll(() => 
    component = StageComponent
      .withResources('src/test')
      .withView('<test foo.bind="firstName"></test>')
      .withViewModel({ firstName: 'Rob' });
  });

  it('does something', done => {
    component.create()
      .then(() => expect(....))
      .then(() => component.bind())
      .then(() => {
           expect(....);
           expect(....);
      })
      .then(() => component.attach())
      ...
      .then(() => component.detach())
      ...
      .then(() => component.unbind())
      ...
      .then(() => component.bind())
      .then(done);
  });

   ...
});

There could also be a component.render() to push it through the standard lifecycle. But the idea is to be able to slowly move the component through it's lifecycle and test at each point.

[sylvain-hamel]

I really like that!

Those thens could be wrappers around setTimeout to make it easy to assert and avoid using setTimeout in the tests like this. I too have to do this a lot in my own tests.

[jdanyow]

you nailed it- that's exactly what the thens are for. A promise's then is async, which will eliminate the need for setTimeout as well as align component testing with standard javascript patterns people are used to.

[stoffeastrom]

Yes it's async but since the UI rendering afaik is happening after the micro task queue you will not always get the desired result depending on what you want to test

[stoffeastrom]

In mocha you could even return the promise chain removing the use of done which is very nice! Don't know if jasmine supports that

[EisenbergEffect]

I'm going to close this issue for now. I've created a new repo where we will put our testing helpers: https://github.com/aurelia/testing It contains a very simple proof of concept. We can handle any further conversation through issues on that repo.

[danielabar]

Is this available as of "aurelia-framework": "npm:aurelia-framework@^1.0.0-beta.1.1.3"?

I'm writing an app that displays a graph where nodes and edges are custom elements. When data changes, the nodes animate into new positions. When a node is clicked on, a border animates in, and click again border fades away. I'm planning on developing more interactions for the nodes and edges, but it's getting difficult to maintain because for each code change, have to click around manually to verify all is still working.

[EisenbergEffect]

It's not quite ready yet. We have a "testing" repo now with some new testing helpers we are working on. I hope it will be ready in the next couple of weeks.

[skeane50]

When unit testing the view is there a way to update the view to reflect a change in the local variables of the view model? Currently I have been calling component.create twice but I was hoping for a better method.