Introduce `aria:` attribute filter

[seanpdoyle]

The Rails provided tag and content_tag methods will transform aria: options into aria- prefixed counterparts.

For example:

      tag.div aria: { label: "Label" }
 # => <div aria-label="Label"></div>

Rails will even JSON-encode Hash and Array instances:

      tag.div aria: { labelledby: ["an_id", "another_id"] }
 # => <div aria-labelledby="an_id another_id"></div>

This commit introduces similar filter-level support for a global aria: filter option to transform Hash instances into compound String keys.

 # => <div aria-labelledby="an_id another_id" aria-selected="true"></div>

 expect(page).to have_css "div", aria: { labelledby: ["an_id", "another_id"], selected: "true" }

[twalpole]

This is Rails specific formatting behavior and doesn't belong in base Capybara

[seanpdoyle]

@twalpole thank you for that guidance. Is there a way to define a global filter?

I'm happy to open a PR to Rails for this diff as well as https://github.com/teamcapybara/capybara/pull/2708, but I'm unsure about what the integration touchpoint would be.

The capybara_accessible_selectors gem defines an aria: expression filter, but it's limited to a filter set that isn't globally applied.

[seanpdoyle]

@twalpole I've opened https://github.com/rails/rails/pull/49665. Outside of feedback on Rails-ism, I'm particularly interested in whether or not it extends Capybara through its "public" API. If you're available and interested, I'd appreciate review on the way it installs the expression filters.

[twalpole]

@seanpdoyle Yes modify_selector is public API, however I don't believe those changes belong as part of Rails either. IMO they should be part or a separate gem users can opt to use or not until it's proven out they're valuable changes and are stable, non problem causing. There's also a chance the aria part could conflict with Capybara in the future which, if included directly in rails, would lock people out of Capybara updates. The data attribute I don't like because it encourages people to test implementation rather than results.

[seanpdoyle]

@twalpole thank you for sharing that guidance.

The data attribute I don't like because it encourages people to test implementation rather than results.

Projects I've worked on have had a lot of success using Capybara selectors outside of System Testing environments. For example, we have Action View and Action Dispatch Integration Tests that use Capybara selectors instead of rails-dom-testing.

In those environments, tests aren't driving the page (or even "visiting" the page) at all, they're strictly making assertions about structure from static HTML without JS or CSS.

Having access to semantically meaningful built-in selectors like :link, :button, etc. combined with the ability to define our own semantically meaningful selectors (or depend on pre-defined selectors like capybara_accessible_selectors) has been immeasurably valuable when compared. The thought process has been: "our test suites already depend on Capybara, and use the same meaningful selectors in browser tests, why not share the same testing language wherever HTML is present"?

In static HTML tests, we've found the higher-level abstractions really valuable (:link, :button, :field), especially when the same assertion can be made with lower level primitives. For example, assert_checked_field "Enable the feature" instead of assert_field "Enable the feature", type: "checkbox", checked: true or even assert_selector :element, "input", type: "checkbox", checked: true.

Having acknowledged that, there are still times where we want to be specific about aria- and data- HTML attributes when a higher-order selector isn't appropriate. This is especially true when writing view and integration tests for applications that utilize data- attribute driven frameworks like Turbo and Stimulus. Having the thorough coverage that asserts that an element has a [data-controller] attribute (or similar) matching a value at the view test layer frees up time and attention to test less rigorously at the browser test layer, where tests are more expensive.

In my experience, applications that use Capybara at the browser test layer should actively discourage asserting particular [data-*] or [class] or [id] attributes, and should instead encourage the higher-order selectors that have end-user facing semantics. Sometimes, [aria-] and [role] attributes are end-user facing, so that can be tricky and usually requires a judgment call instead of a blanket rule.

All this is to say that Capybara has become the most important dependency on most Rails applications that I work on that server-generate HTML.

That isn't hyperbole. The fact that selectors can be available at every layer combined with it's sophistication make Capybara irreplaceable. In spite of the tremendous value we get out of using the tool in this way, I have the feeling that how we've utilized the selector system is at odds with Capybara's stated goals, and that we're pulling in opposite directions. The underlying assumption is that Capybara prioritizes browser environment tests above all else, and that trying to use it outside of that environment isn't explicitly supported (with the exception of "fast" System Tests with Rack::Test).

Lately, I've wished for a version of Capybara that exists on a continuum between the gem as its structured today and a gem like teamcapybara/xpath.

Is there any interested in making it possible to consume Capybara in separate layers?

• the layer that abstracts over Nokogiri with XPath and its selector system
• the layer that abstracts over Selenium / Webdriver / etc with its browser-driving behavior

If not, is there interest in improving support for Capybara in browser-less environments?

There's also a chance the aria part could conflict with Capybara in the future which, if included directly in rails, would lock people out of Capybara updates

This would be bad! If there was a way to make the selector-only package extensible, that's something that third-party packages might help de-risk.

[twalpole]

I believe what you've written reinforces that the changes/enhancements you're pushing for should really be in their own gem which people would then be free to include into the projects if they want the same functionality you're going for rather than forcing it onto people in rails

[seanpdoyle]

@twalpole thank you for that feedback.

In spite of the tremendous value we get out of using the tool in this way, I have the feeling that how we've utilized the selector system is at odds with Capybara's stated goals, and that we're pulling in opposite directions.

Am I correct in my understanding of where Capybara sees itself fitting into a broader test suite?

the changes/enhancements you're pushing for should really be in their own gem

Is Capybara::Selector part of the gem's public interface? Is Capybara::Selector.all a reliable way for third-party gem's to extend Capybara, or is that breaching layers of encapsulation?

[twalpole]

I see Capybara providing the most value with browser based tests. The selectors can be used in view tests, and are useful there, however I see browser based behavior tests being much more valuable than view tests. Testing that views have specific data- values in elements is meaningless if you're not actually testing that those data- values are correctly used by the JS in the browser, and if you are testing the behavior in the browser, what's the point of testing they exist in the view?

Yes Capybara::Selector is public interface, although I wouldn't use Capybara::Selector.all to loop through and modify all the selectors unless you give people a way to opt out of modifying specific selectors. There's way too high a chance of conflicting with peoples custom selectors. As a user I think I would actually prefer an opt-in per selector configuration.