whatwg / html

HTML Standard
https://html.spec.whatwg.org/multipage/
Other
7.98k stars 2.61k forks source link

Improve DX of element reference attributes by allowing relative references instead of only ids #10143

Open LeaVerou opened 6 months ago

LeaVerou commented 6 months ago

Problem statement

Many HTML attributes need to reference one or more HTML elements in the document. This includes:

It is also a frequent use case in author web components as well, with WC authors coming up with wildly inconsistent solutions because they are forced to choose between ergonomics and consistency with the web platform.

Currently, the only way to specify such references is to give these elements ids (if they don't already have them) and using these ids to link to them in these attributes.

This imposes high friction (especially when not using tooling to generate HTML), as authors then need to come up with suitable ids for elements that wouldn't otherwise have one and manually do the linking (and as we know, naming things is one of the hardest things in Computer Science 😁 ). It also introduces error conditions, as it is a common authoring mistake to change an id and forgetting to change the id references to it, pasting a chunk of HTML and forgetting to edit all the references, or ending up with broken references due to accidental duplicate ids.

This is a very common author pain point, and authors are pretty vocal about it: DX-related complaints were the 3rd biggest a11y complaint in the preliminary State of HTML results. It especially hurts a11y, since the effects of broken references in the a11y tree are not always obvious, and the more friction it takes to make HTML accessible, the less likely authors are to do it. And while for <label> this is somewhat mitigated by the option to make the association implicitly by nesting the form control within the <label>, for the other cases there is no similar option.

Being able to link to elements in a way that is relative to the element the attribute is specified on would solve all of these issues, and make writing ARIA much more pleasant.

Some considerations are:

Relative reference use cases

More research is needed here, but in my experience most relative use cases are pretty simple paths from the current element to the one being referenced. Things like:

  1. Go N levels up (usually 1-2) and get the first element with a class of foo
  2. Get the next/previous element

Proposed solutions

There are two components to coming up with a solution:

  1. Syntax: How to specify elements relative to the current element (i.e. the one the attribute is specified on)?
  2. Disambiguation: Assuming we don't want to add new attributes for this, how to disambiguate this syntax from the id references currently in use?

1. Disambiguation

In theory IDs can contain any character, though in practice they very rarely contain characters going beyond CSS idents. So how do we come up with a syntax that minimizes conflicts with ids used in the wild? There are two main categories here.

1a. Syntactic switch

This approach allows mixing absolute and relative references even on the same element by using a syntactic switch to say "this is not an id, it's a relative reference".

It would require a fair bit of web compat research to flesh out the details (I can reach out to the HTTP Archive folks), but the main paths here are:

  1. Imposing restrictions on the relative syntax so that it needs to contain certain characters that are very unlikely to appear in IDs, e.g. that the value needs to begin with & or :. Ids can still be specified by escaping these characters.

Example:

<label for="& + *">Foo:</label>
<input name="foo" />
  1. A functional syntax that wraps the relative reference (e.g. selector(), path(), relative(), ref() etc.). This is more verbose, but has the added benefit of clarity and extensibility. If plain names like selector() are not web-compatible, we could go the route of URL fragments and prepend these functions with a certain symbol to further minimize the odds of collision.

Example:

<label for="selector(+ *)">Foo:</label>
<input name="foo" />

1b. Scoping attribute

Instead of a special syntax, this would introduce an additional attribute that switches how references work on an entire subtree.

Ideally, the attribute is not just an opt-in, but also adds value, e.g. by specifying the scope of matching so that references can be simplified. Scopes can be nested, and the parent scope is matched if the closest scope did not yield any results. The syntax of individual attributes need to provide ways to escape the scope, for the use cases where global matching is genuinely desirable.

A big downside of this approach is that because it affects references across a whole subtree, it makes migration more painful, unless we do weird things like "match as an id first, and if that doesn't match anything, try something different", which can be unpredictable and error-prone.

Example:

<li idscope>
    <label for="foo">Foo:</label>
    <input id="foo" />
</li>
<li idscope>
    <label for="foo">Foo:</label>
    <input id="foo" />
</li>

2. Syntax

I see two avenues here:

  1. CSS selectors, potentially with severe restrictions, especially at first.
  2. Identifiers that an already be used on more than one element (e.g. class or name)

An attribute to restrict scope (see 1b) would be useful for both, but while it is a convenience for 1, it is essential for 2 to be useful.

While a custom microsyntax might be tempting, I would advise against it (we even have a TAG principle in the works advising against custom microsyntaxes).

2a. CSS selectors

CSS has recently introduced relative selectors that start with a combinator and/or can use :scope or & to represent the current element (see 1 2). If relative selectors could be allowed in these attributes, authors could do things like + .description or .description:has(+ &) etc. If the selector specified matches multiple elements, the first one will be used unless the attribute expects multiple elements.

Not the entirety of CSS selectors needs to be allowed. In fact, I think an MVP could be as small as just <id-selector> | [<combinator>? [ <type-selector> | '*']? <class-selector>* ]+ (see below wrt combinators). In syntaxes that involve a scoping attribute (see 1b), <id-selector> could still match globally, providing an escape hatch from the scoping.

Pros:

The main downside seems to be performance. Having to specify "the previous element" with a :has(+ &) that searches the entire DOM tree is likely unacceptable. I would need to check with the rest of the CSS WG, but I suspect that if the WHATWG is interested in pursuing this, we might be open to exploring combinators that go backwards (previous sibling, parent) to facilitate common cases without :has(). Things like - for "previous sibling" and < for parent have been proposed before and add value to authors more generally as well. I suspect this would only be tenable if these kinds of backwards combinators are a possibility and/or combined with an attribute to limit scope.

2b. Identifiers

Even something as restricted as being able to specify an identifier and a root for the query would address the vast majority of use cases. So basically all we need is a way to express a non-unique identifier and a scoping attribute to mark the root of the matching.

Candidates for this could be:

This produces a very concise, readable syntax for common cases, maintains the same syntax for the individual attributes, and removes the disambiguation problem. However, it is unclear whether a single hierarchy of scopes would be sufficient to cover use cases, and makes it harder to interpret individual values.

There are two strategies here, each with its own tradeoffs.

Only look at non-unique identifiers in subtrees with the scoping attribute

This means authors would have to opt-in to this kind of matching, by using a scoping attribute (e.g. namescope, or even something that specifies the identifier being scoped e.g. refscope="name").

This is more predictable, but makes migration costly. To enable referencing elements globally, the scoping attribute would need to be added to the root element as well, which makes it even harder to mix and match id references with relative references.

Match as id first, fall back to non-unique identifier if no element found with said id

This ensures that copying a chunk of HTML within another does not break, but it means that references can break by simply adding an id in another place in the document, which can be very hard to debug. There is also no precedent in the web platform where the same identifier can mean either id or something else (especially with different scopes!).

3. What if we could fix this without any new syntax?

This is a stretch, but might be worth exploring. name is an existing attribute that identifies an element similarly to an id, but does not have the restriction of uniqueness.

The algorithm for resolving references could be redefined as:

  1. First, look for an element with that id. If found, return that.
  2. Let scopingRoot = referencing element
  3. While scopingRoot != document
    1. Look for an element with a name equal to the identifier provided inside scopingRoot.
    2. If found, return it.
    3. Otherwise, let scopingRoot = parent of scopingRoot

I wonder how web-compatible something like that would be. Since it would only make a difference if the reference is broken to begin with, maybe it's not too unrealistic?


I’m still unsure what the best solution is, but I’m leaning towards 3 if it's web-compatible, or 1a + 2a if not, perhaps with an optional scoping attribute.

Rationale:

If there is consensus to pursue this, I could do the research of exploring what syntax could be web-compatible.

sashafirsov commented 6 months ago

The scope perhaps can be less demanded when the root element for web components become a thing beyond CSS styling. But even with that, the relative selector is highly needed.

The selectors language perhaps need to be addressed in another proposal. As URL can have the protocol, selectors should as well. CSS and XPath are natively supported by most of browser. Others or extensions of base ones would be polifilled.

LeaVerou commented 6 months ago

As URL can have the protocol, selectors should as well.

They already do, as I discuss in the first post. 🙃 The syntax discussion here was about what subset of relative selectors to support, not to define relative selectors.

triskweline commented 6 months ago

I can share some pains, design decisions and experiences we had with relative references over the years. This is from my work with Unpoly, an HTML extension where links update fragments instead of full pages. For this we needed a way to reliably target elements.

We have not needed introduce new syntax aside from :origin and :has() (before browser support was there). However, a similiar library htmx uses custom selector synxtax like closest .foo.

jimmyfrasche commented 6 months ago

This is another area where it makes it hard to reason about server side template partials like I discussed re hN tags in https://github.com/whatwg/html/issues/5033#issuecomment-547138261

Another problem with id other than uniqueness is that it does way too much.

I would like something like this sketch:

A refscope tag (or maybe it's an attribute) defines a new scope. :root implicitly defines a root scope.

Elements have a ref attr to use instead of id. Unlike id it only defines a name and that name is local to the current scope. (If there are multiple identically named refs, then which one wins should be defined in the standard and browsers should print a warning in the debug console.)

If something is referred to by name, the refs in the current scope are searched first. If there's no match, check the parent scope; repeating as necessary until reaching the root scope. If there's no match after all that, check for a matching id.

A tag could have both a ref and id and that may be confusing and certainly not recommended but everything works out.

dginev commented 6 months ago

The MathML 4 effort has some overlapping interest in improved DX for referencing. Exactly due to the difficulties of managing global ids, we have tentatively introduced a new arg attribute to anchor referencing in MathML Intent, targeting the feature of accessible navigation of math expressions: https://w3c.github.io/mathml/spec.html#intent_reference

Simple example:

<msup intent="power($base,$exp)">
  <mi arg="base">x</mi>
  <mi arg="exp">n</mi>
</msup>

We have had the constraint of keeping the traversal algorithm as simple as possible, which has so far kept us from introducing relative selectors (sibling, ancestor). Nevertheless, MathML has viable use cases where relative selectors are quite natural, such as accessible walks through tabular diagrams (realized via <mtable>).

The traversal algorithm for intent/arg referencing is scoped to the current subtree (references can only point to descendant nodes). It also has an additional stopping condition at nodes carrying the intent or arg attributes, to allow stacking of multiple math notations.

As we have multi-format content (such as scientific diagrams via SVG+MathML, as well as block equations mixed with text), it would be a significant boost to DX if we could share a uniform referencing scheme.

sorvell commented 6 months ago

Feedback

@LeaVerou

Many HTML attributes need to reference one or more HTML elements in the document

It may be worth distinguishing the problems describing 1-to-1 relationships with those describing 1-to-N. For example, ids are not used for <input type="radio"> but are, unfortunately, required for things like aria-controls.

Potential to solve more than ergonomics

I suggest this be promoted to a core goal of the proposal. This is a bit spicy but arguably the current lack of a way to associate elements across scopes violates platform design principles.

Being able to link to elements in a way that is relative to the element the attribute is specified on would solve all of these issues, and make writing ARIA much more pleasant.

I think the problem isn't clearly enough defined to determine if this is true. In particular, the id based system is independent of document structure, and most of the ideas here for a relative system depend on it. That just seems like a trade off that may be better or worse for the various different use cases.

Proposed solutions

The various options here are a little hard to understand. It'd be great to add some illustrating examples.

@jimmyfrasche

Another problem with id other than uniqueness is that it does way too much.

Totally agree, and this may even be the core problem to solve.

Thinking Aloud...

LeaVerou commented 6 months ago

I just added another proposed idea to the first post:

3. What if we could fix this without any new syntax?

This is a stretch, but might be worth exploring. name is an existing attribute that identifies an element similarly to an id, but does not have the restriction of uniqueness.

The algorithm for resolving references could be redefined as:

  1. First, look for an element with that id. If found, return that.
  2. Let scopingRoot = referencing element
  3. While scopingRoot != document
    1. Look for an element with a name equal to the identifier provided inside scopingRoot.
    2. If found, return it.
    3. Otherwise, let scopingRoot = parent of scopingRoot

I wonder how web-compatible something like that would be. Since it would only make a difference if the reference is broken to begin with, maybe it's not too unrealistic?


Thanks for the thoughtful response @sorvell. Some replies:

Many HTML attributes need to reference one or more HTML elements in the document

It may be worth distinguishing the problems describing 1-to-1 relationships with those describing 1-to-N. For example, ids are not used for <input type="radio"> but are, unfortunately, required for things like aria-controls.

Interesting. What do you think are the differences?

Potential to solve more than ergonomics

I suggest this be promoted to a core goal of the proposal. This is a bit spicy but arguably the current lack of a way to associate elements across scopes violates platform design principles.

As an editor of the document you are referencing, I have no idea how that principle relates to this problem. 😁 That said, I'd be fine with making that a core goal of the proposal, though I'm not sure how that will materially affect the odds of it moving forwards.

Being able to link to elements in a way that is relative to the element the attribute is specified on would solve all of these issues, and make writing ARIA much more pleasant.

I think the problem isn't clearly enough defined to determine if this is true. In particular, the id based system is independent of document structure, and most of the ideas here for a relative system depend on it. That just seems like a trade off that may be better or worse for the various different use cases.

Why is it a goal to have a referencing system that is independent of document structure? Document structure is not created in a vacuum, it typically reflects hierarchical relationships (and when it doesn't, that's a failing of some web platform technology — e.g. when authors have to put elements directly inside <body> to make sure they're on top).

Also, the id-based system is not going away. That wouldn't be web compatible. Any relative referencing system is in addition to it.

Proposed solutions

The various options here are a little hard to understand. It'd be great to add some illustrating examples.

Agreed. I added a few examples, does this help? I could add more if you point me to specific sections that are still unclear.

  • Naming things is hard, but uniquely naming things can be really problematic since it requires global knowledge.

+1000

Where this isn't necessary, it shouldn't be required. This is already the case for <input type="radio" name="a"> since these are form scoped.

  • There are currently often special attributes for one side of the equation only, e.g. for, popovertarget, invokertarget. Perhaps the other side should have them as well: e.g. popoverid, etc.

Naming things is hard so let's add more naming tasks? 😁 What are some use cases where you may want to have a different name for e.g. popover use cases and a different one for e.g. forms?

  • If the tree is used for scoping, explicitly using a scoping element may be worth it for clarity.

Unless any ancestor container could potentially be the scope, and proximity determines where to look, see new option 3.

  • Shadow DOM is an existing DOM scoping mechanism, can it be used as part of the solution here? If not, what abilities might it need to gain to be useful for this?

Shadow DOM is a very heavyweight scoping mechanism that does too much. Nobody would switch to Shadow DOM simply to use relative references, as the friction involved in that far, far outweighs the friction of assigning unique ids.

mayank99 commented 2 months ago

A potential 4th (or 2c) option that might play nicely with HTML modules, DOM parts, and template instantiation: "Automatic generation of unique ids in each instance of the template".

Consider this template that uses the declarative DOM parts syntax (which might change):

<template>
  <label for="{{ uid(fruit) }}">Fruit</label>
  <input id="{{ uid(fruit) }}" />
</template>

Here, uid() is like a built-in function that's available for use within DOM parts. It takes an optional key that is scoped to the template, and returns an IDREF that is guaranteed to be globally unique.

When instantiated, this template would produce something like:

<!-- First instance -->
<label for="fruit001">Fruit</label>
<input id="fruit001" />

<!-- Second instance -->
<label for="fruit002">Fruit</label>
<input id="fruit002" />

The optional key is only useful when you need to actually associate it with another element. It can be omitted when there's a simple need for a unique IDREF.

<template>
  <div role="option" id="{{ uid() }}">{{ }}</div>
</template>

<!-- instantiated N times -->
<div role="option" id="__001">Option 1</div>
<div role="option" id="__002">Option 2</div>
<div role="option" id="__003">Option 3</div>

This probably wouldn't solve all cases, but I just wanted to put it out there. To me, it feels more flexible than something like idscope because it's explicitly opt-in (meaning you can still use global ids in the same template). It is new syntax, but we'll need that new syntax for DOM parts anyway.

dbaron commented 2 days ago

So while XPath hasn't gotten much interest lately, it's possible that it might be a good fit for this sort of thing.

The reasons I think XPath may be worth considering here are:

  1. XPath already has solid syntax for relative paths starting from somewhere in the middle of a tree
  2. I think CSS selectors and XPath are optimized for slightly different problems:
    • CSS selectors were optimized for the problem of "given one element, find all the selectors that match"
    • XPath was optimized for the problem of "given an XPath expression, find all the elements that match"

It's perhaps questionable whether it falls into the "another microsyntax" trap: it's already a part of the Web (see e.g. domxpath and dom/xslt tests), but most of the things that use it are not widely used today. There's definitely a learning curve for those who don't know it -- but it's also pretty powerful and likely to do a pretty large subset of the things people would want from a feature like this.

[Edited to add] A big caveat that would need some research is that I'm not sure how well the existing implementations work with HTML (rather than XML).

sashafirsov commented 2 days ago

+1 @dbaron XPath is acually part of browser DOM implementation via document.evaluate method. Which makes in a native feature to be extended to HTML attribute as in this proposal. There is a DCE (Declarative Custom Element) implementation which is using XPath for such kind of relations.

Even if the attribute can refer either ID, or css, the URI convention with schema as xpath: preffix will fit there well.

Delapouite commented 2 days ago

Talking about XPath, a few years back @WebReflection attempted to get the ball rolling around upgrading the spec: https://github.com/whatwg/dom/issues/903

WebReflection commented 1 day ago

if there's anything I could add to that old attempt to refresh XPath on the Web:

That being said, when it's @LeaVerou suggesting DX improvements, I am usually 100% in favor of that so I won't bother this thread anymore with some hope that some solution would land in a way or another 👋

jimmyfrasche commented 1 day ago

Any query language—css, xpath, or something new—is going to have the same problems. It's going to be brittle and hard to use because you have to specify a path through the DOM and then the DOM changes either due to interaction with js or just during maintenance, etc.

This is job for lexical scoping. We just need a way to say "this subtree is a new scope" and "this name is scoped" and use a scoped name. For most cases you could just start a new scope on the outermost element of a partial template or on your component boundary. Those could still be changed, of course, but they're clearly marked so at the very least you know to think "hey, I should be careful here, this will affect name resolution"

There are certainly things you could do with a more powerful mechanism but nothing that would be easy to understand or maintain so it would end up being one of those things where the advice is to ignore everything it can do and just stick to some magic patterns copy pasted so many times they're essentially oral tradition.

I'm going to take my previous post and 1a and 1b in the first post and mix them up:

  1. a refscope attribute introduces a new scope
  2. a ref=name attribute names an element in the current scope
  3. $name instead of name means look up a ref in the current scope instead of an id in the global scope

Since this system is independent of the old id system it can be polyfilled by adding a globally unique id to everything with a ref then remapping all $name references in scope to that id.

(or some other sigil if $ can't be used)

WebReflection commented 1 day ago

introduce a ref attribute on the Web and good luck explaining it to the plethora of the JSX users out there ... also, moar attributes doesn't seem to fix the underlying limitation problem, but I would, personally, think of a better name than ref. As usual, my voice is rarely heard in this place, worth "yelling at clouds" though in this specific case.

jimmyfrasche commented 1 day ago

The name is unimportant. It can be whatever works. ref is just the first thing I thought of and I need something to express the idea.

WebReflection commented 1 day ago

The name is unimportant

please remind me when that has ever been the case in programming ... but fair enough, I just wanted to voice the majority of the JS community I don't even belong to (JSX / React) but if your one is the solution, and I almost agree with your preface, even if Xpath is already there and embedded in specs, not something new, please consider anything different from ref as attribute name or see your solution doomed into developers' confusion, thank you!

jimmyfrasche commented 1 day ago

I mean that a better name can be worked out later before it is specced and implemented in browsers. Consider ref a placeholder.

sashafirsov commented 1 day ago

The purpose of CSS is not to define the relations in the dom structure, XPath subset is specifically designed for it.
The decision of reject XPath on CSS level should not impact the reference attribute IMO.

As for scoping, this is a complete own aspect which should not be part of this discussion. Partially it is a part of scoped components registry. Definitely the ability to define the "scope" in the dom sub-tree and then reference it from (XPath) query attribute would be beneficial. But would need a separate thread on how to handle cross-scope barriers. Would love to participate such discussion on another thread.

tabatkins commented 1 day ago

Both XPath and Selectors do the same thing, just with slightly different powers due to historical circumstance. XPath is a little stronger on crawling the tree, Selectors is a little stronger on querying individual elements. That's it; they're otherwise identical technologies at a slightly abstract level.

The downside of using XPath is that it's a second querying language, similar to but different from Selectors, and their behavior and functionality largely overlap. The Web already contains a profusion of languages, and we should be careful with adding more. (Yes, XPath is technically exposed via one DOM method already, but it's little-known.) It also means the large variety of element qualities that Selectors can match on wouldn't be available; we'd be restricted to the much more limited set of things XPath can inspect.

The downside of using Selectors is that we might need to add more combinators to handle traversing along additional axises beyond just "descendants" and "following siblings", and these wouldn't be valid in CSS for perf reasons. But we have the theoretical structure for this (/foo(...)/ combinators), so maybe it wouldn't be too bad.

jimmyfrasche commented 1 day ago

I can visualize solving all the problems I've run into by using the scoping mechanism I posted earlier. I can't think of anything that would be simpler or better using a query language. If I had a query language instead of scoping I would essentially reinvent most of scoping by having all queries be "go up to the last element tagged as a boundary then go back down and find the specified name without entering any other tagged boundaries"