New assertion API

[jamesshore]

Update: See this comment for my specific proposal.

Original comment follows...

---

Quixote's assertion API is a bit... clunky.

element.assert({
  width: 10,
  height: 20
});

The original intent was that you would be able to make multiple assertions about an element quickly. But in practice, you're often just asserting one thing about an element. Also, equality isn't the only kind of assertions you might want to make.

So a more-traditional assertion mechanism that allows one-liners would be nice.

element.height.should.equal(20);
element.width.should.be.lessThan(10);

It would also be nice if it supported multiple assertion flavors.

element.height.assert.equals(20);
element.width.assert.lessThan(10);

And of course, we want to keep our high-quality error messages and relative assertions.

element.height.should.equal(body.height);
// Outputs:
// height of '#element' was 10px smaller than expected.
// Expected: 50px (height of 'body')
// But was:  40px

However, people have their preferred assertion frameworks (such as Chai), and it would be nice to support those as well.

var assert = require("chai").assert;
var expect = require("expect");

assert.equals(element.height, body.height);
expect(element.height).to.equal(body.height);

Share your thoughts about improvements to Quixote's assertion API in this issue.

[JuanCaicedo]

I definitely agree with the last part about wanting to be framework agnostic. It's would be a shame to lose the nice error messages though. I'm not familiar with what API's tools like Chai expose, but maybe there's something Quixote could provide to interface with them and keep error messages.

One thing I got to thinking about reading src/values/readme.md, particularly this line

Descriptors represent some as yet uncalculated aspect of CSS

Is that maybe the right interface for a descriptor is a promise. Promise are values that eventually resolve to a value. This means that if you use a framework like Chai, you could use standard promise testing tools like chai-as-promised

expect(element.height).to.eventually.equal(body.height);

Then Quixote could resolve promises under the hood and its own API wouldn't need to change.

This might not be necessary, but I wanted to point out that similarity in case it is.

[jamesshore]

Promises are superficially similar, but not what we need in this case. A descriptor represents some part of the page and provides the ability to calculate its value synchronously, on demand. A promise represents an asynchronous calculation in progress that will eventually resolve to a value.

[jamesshore]

While we're thinking about improvements to assertions, I also think Quixote's messages can be improved. Currently, the summary says how the result was different than expected:

height of '#element' was 10px smaller than expected.
Expected: 50px (height of 'body')
But was:  40px

But in practice, I find this confusing. When a test fails, I want to see what I need to fix. The test could say that instead:

height of '#element' should be 10px larger.
Expected: 50px (height of 'body')
But was:  40px

There may be opportunities to improve the grammar, as well:

Instead of: "height of '#element' should be 10px larger"
Use: "'#element' height should be 10px larger"
Or even: "#element height should be 10px larger" (no quotes around '#element')

[jamesshore]

Okay, I'm getting ready to implement a new assertion API for the next release of Quixote. Here's what I'm currently thinking. Feedback appreciated! (Summoning @woldie.)

As a reminder, Quixote works by allowing you to make assertions about the visual layout of your elements and pages. You're not making assertions about specific CSS properties, but instead about how your CSS affects where elements are rendered on the page.

Overview

Most objects will have a should property that can be used to make assertions. They'll all have these two assertions:

• [x] should.equal(...)
• [x] should.notEqual(...)

For example:

elementA.left.should.equal(20);    // left edge of elementA should be at X-coordinate 20
elementA.right.should.equal(elementB.right)    // right edge of elementA should be same as right edge of elementB

The above is equivalent to the following code in our existing API:

elementA.assert({
  left: 20,
  right: elementB.right
});

However, it's more flexible and (I hope) easier to read and understand.

In addition to the baseline equal and notEqual assertions, there would be a lot of situation-specific assertions:

Positions

Positions are things like element.left and element.top.

• [x] should.beAbove(position)
• [x] should.beBelow(position)
• [x] should.beToRightOf(position)
• [x] should.beToLeftOf(position)

If the above assertions work well, we could add these as well:

• should.beAtLeastAsHighAs(position) AND/OR? should.notBeBelow(position)
• should.beAtLeastAsLowAs(position) AND/OR? should.notBeAbove(position)
• should.beAtLeastAsFarRightAs(position) AND/OR? should.notBeToLeftOf(position)
• should.beAtLeastAsFarLeftAs(position) AND/OR? should.notBeToRightOf(position)

Sizes

Sizes are things like element.width and element.height. But they're also used for distances, such as a.right.to(b.left). (Distances were added in v0.13.)

• [x] should.beBiggerThan(size)
• [x] should.beSmallerThan(size)

If those work, these are also options:

• should.beAtLeastAsBigAs(size) AND/OR? should.notBeSmallerThan(size)
• should.beAtLeastAsSmallAs(size) AND/OR? should.notBeLargerThan(size)

Elements / Rectangles

Additional assertions could be added for elements. They weren't included with the first batch, so they've been documented in issue #58.

Elements—or perhaps any object that has a top, bottom, width, height, etc.—could have shortcut assertions for a lot of the position and size assertions, such as:

• should.beAbove(element)
• should.beWiderThan(element)
• ...etc...

But they could also have some more interesting positioning assertions, such as:

• should.notOverlap(element)
• should.overlapTop(element, optionalSizeToOverlap)
• should.overlapBottom(element, ...)
• should.overlapRight(element, ...)
• should.overlapLeft(element, ...)
• should.abutTopOf(element)
• should.abutBottomOf(element)
• should.abutRightOf(element)
• should.abutLeftOf(element)

Frame

This is where things get really interesting. We could make assertions about how elements are aligned:

• should.centerAlign(element, size, element, element)
  • This would say that all the elements have the same center. If there was a size in there, then every element after that size would be expected to be that wide.
  • For example: frame.should.centerAlign(navbar, logo, content, footer)
  • AND/OR? perhaps the sizing and alignment should be separated. We could have should.matchWidths(optionalSize, element, element, element) as well.
• should.leftAlign(...)
• should.rightAlign(...)
• should.middleAlign(...)
• should.topAlign(...)
• should.bottomAlign(...)

We could also make assertions about how elements are arranged from left to right, or top to bottom:

• should.flowHorizontally(element, size, element, size, element)
  • This would say that the elements proceed one after another, with size gaps in between.
  • For example: frame.should.flowHorizontally(columnA, 10, columnB, 10, columnC) means that columnA is on the left, then there's a 10px gap, then columnB, then a 10px gap, then columnC.
• should.flowVertically(...)

Feedback Wanted

• Do these assertions make sense?
• Do they seem useful and powerful?
• Where I said AND/OR, which of the two (or both) is your preference?
• Which ones are overkill, or over-complicated?
• Which commonly-needed assertions am I missing?

[woldie]

This is a nice proposal! Would you consider adding one nesting level after the should? Something like should.flow.horizontally(...)

That way we can reduce our camel-casey names and keep more of the things all lowercase.

Also I think any rectangular object should sport the *Align functions, not just Frame.

[jamesshore]

I'm not opposed to additional nesting levels, and I don't want to create confusion about when to use a dot and when to use camel-case. Can you list out what it all the above assertions would look like in a two-level scheme?

Agreed on the align functions--in fact, I'm not sure if they should be on Frame at all.

[jamesshore]

A couple of quick thoughts before bed. First, I want to see what @woldie's two-level nesting looks like. Second, I think the element/rectangle assertions are going to be the most powerful. We'll drop down to specific width and position assertions at times, but often we'll be working with elements.

Let's say we're testing a cookie notification bar. (Everybody loves cookie notifications, right? Right.) It's pasted to the bottom of the window, full-width, and is 20px tall.

Old assertion style:

var viewport = frame.viewport();
cookieBar.assert({
  width: viewport.width,
  left: viewport.left,
  bottom: viewport.bottom,
  height: 20
});

New style, low-level assertions

var viewport = frame.viewport();
cookieBar.width.should.equal(viewport.width);
cookieBar.left.should.equal(viewport.left);
cookieBar.bottom.should.equal(viewport.bottom);
cookieBar.height.should.equal(20);

New style, high-level assertions:

var viewport = frame.viewport();
cookieBar.should.beInside(viewport);
cookieBar.should.align.bottom(viewport);
cookieBar.width.should.equal(viewport);
cookieBar.height.should.equal(20);

Or perhaps?

cookieBar.should.fill.bottomOf(frame.viewport());
cookieBar.height.should.equal(20);

I don't think I've got it 100% right yet. I want the API to be as solid and unchanging as possible once released. It deserves more thought. What other scenarios should I create examples for?

[jamesshore]

Element assertion ideas so far (plus a few new ones). Element.should...

• element.should.align.[top | bottom | left | right | middle | center](elementB, ...) (share the same edge with elementB and other elements provided)
• element.should.align.[vertically | horizontally](elementB, ...) (aligned with, and same width/height as, elementB)
• element.should.abut.*(elementB, ...) (share opposite edges with elementB; perhaps also touch a little bit?)
• element.should.contain(elementB, ...) (elementB is entirely inside element)
• element.should.beInside(elementB) (element is entirely inside elementB)
• element.should.fill.[allOf | topOf | bottomOf | leftOf | rightOf | horizontalStripOf | verticalStripOf](elementB) (element touches 2-4 edges of elementB)
• element.should.beFilled.[entirelyBy | fromTop | fromBottom | fromLeft | fromRight | horizontallyBy | verticallyBy](elementB,...) (opposite of .should.fill, all but entirelyBy allows multiple elements) (perhaps should be on element.should.contain instead)
• element.should.center.[in | horizontallyIn | verticallyIn](elementB) (centering)
• element.should.flow.[horizontally|vertically](elementB...) (elements proceed one after another, with gaps in-between)

And perhaps we need a way of grouping elements into a larger rectangle, without assuming they're in a div. For example, we might say that three elements are all in a perfect row, and that row is centered in another element.

Just some ideas that need further thought. Still looking for scenarios--please share!

[jamesshore]

I started implementing the new assertion API a few days ago and it's coming along very well. I've already implemented should.equal() and the new error messages described in this comment. I'll track my progress in this comment.

[woldie]

Finally doin' it, eh? Took a pandemic to make it happen. 😷

I'm enjoying Google's Truth assertion library lately, might be an additional fount of inspiration worth drawing from.

On Sun, Apr 12, 2020, 6:52 PM James Shore notifications@github.com wrote:

I started implementing the new assertion API a few days ago and it's coming along very well. I'll track my progress in this comment https://github.com/jamesshore/quixote/issues/47#issuecomment-345486461.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/jamesshore/quixote/issues/47#issuecomment-612713570, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACOGQYCVJLA7O7DDTF7T7CLRMJV7JANCNFSM4DH7DGVQ .

[jamesshore]

I've finished the first batch of assertions for inclusion in v1.0, so I'm tagging this issue "done." The remaining assertions have been moved to issue #58 for work at a future date.

Although this is issue is done, I'm leaving it open until v1.0 is released.

[jamesshore]

Released in v1.0.0.