Generic programs can't reliably use/manipulate documents via the DOM

[mrmr1993]

Alternative title: DOM does not reliably model document objects with closed shadow DOMs.

A generic program (ie. a program that uses the DOM to interact with a document of which it has no prior knowledge) has no reliable interface with which it can interact with and/or monitor documents. In particular:

• the true active element in the document cannot be discovered when inside a closed shadow DOM. Thus,
  • the program cannot reason about the default action associated with eg. keydown, keyup and click events, and so cannot predict behaviour in response to user input or its own simulated input.
  • the program cannot reliably direct or redirect the focus of the page in order to perform further actions
  • a selection cannot be read (except as a string by Selection.toString) or modified (except by the non-standard Selection.modify)
• the contents of the shadow DOM are completely inaccessible to the program.
  • Since the program has no specific knowledge of the document, it cannot reason about it in any reliable way.
  • Further -- even when the document/component provides an API for (some of) its contents -- the program has no way of knowing that it exists or how to use it.
  • A user may expect (or even direct) the program to interact with a closed shadow DOM child, since there is no perceptible difference from a normal element to them.
• a program with access to the DOM but not the document's Javascript context cannot use any of the usually suggested techniques to circumvent this. Its only option is to proceed with broken or incomplete functionality.

Closed shadow DOMs may (or may not) be fine for document authors, who make the decision to use code including them. However, their existence and use makes the DOM an unreliable API for programs, libraries, etc. that are made to act upon a document by the user or document author.

If the DOM is supposed to be an effective API for developers other than document/component authors, then closed shadow DOMs are directly harmful (edit:) to all these developers. I propose depreciating mode and dropping closed shadow DOMs completely.

This issue generalises #354, all examples there also apply to here. (/cc @dylanb in case you're still interested).

[dylanb]

If the DOM is supposed to be an effective API for developers other than document/component authors, then closed shadow DOMs are directly harmful. I propose depreciating mode and dropping closed shadow DOMs completely.

I think this statement is too generic. Closed shadow DOM is VERY useful in masking the author from the unnecessary details of the internals of standard HTML elements but harmful otherwise.

[mrmr1993]

Closed shadow DOM is VERY useful in masking the author from the unnecessary details of the internals of standard HTML elements

I think the "standard" part here is important: how is any code not written by the document author supposed to deal with the myriad possible non-standard elements otherwise? The specs dictate that the standard elements should behave in the manner anyway, with or without a closed shadow DOM feature.

I'll concede an edit, regardless.

[rniwa]

A generic program (ie. a program that uses the DOM to interact with a document of which it has no prior knowledge) has no reliable interface with which it can interact with and/or monitor documents

What are examples of such a library / framework / program? The only concrete example people have presented so far is Google Feedback (it creates a screenshot of a Google property in JS). But any first-party feature like that could assume that everyone uses the same library or even a convention used on websites, e.g. shadowRoot is always exposed via _shadowRoot(), to implement such a feature. So the case in which components are written by the first party isn't the interesting case.

Now, let us consider the case where components are written by third party developers, or different teams within an organization with tens of thousands of developers. In this case, it's not trivial to write a code that walks across shadow trees of components written by different teams or third parties. However, this can also be a benefit. If one of those teams decide to change the internal DOM structures of a component, any JS code that relies on certain DOM structure in that component's shadow DOM would break.

In fact, this is a feature of Web Components API since the intent of Web Components API is to let developers write a custom HTML element that hides its internal structure in favor of providing a public API like any builtin HTML element. If it was so easy to travers and interact with each component's shadow tree, it would defeat the whole point of encapsulation. Instead, any generic code that walks across DOM to do some work should be treating each custom element instance like a regular builtin HTML element.

[rniwa]

I think the "standard" part here is important: how is any code not written by the document author supposed to deal with the myriad possible non-standard elements otherwise? The specs dictate that the standard elements should behave in the manner anyway, with or without a closed shadow DOM feature.

The same we'd deal with video elements, input elements, etc... which can be modeled as builtin elements with UA shadow roots (WebKit and Blink DO use shadow roots to implement these elements).

[mrmr1993]

What are examples of such a library / framework / program?

My classic examples are:

• browser extensions (specifically WebExtensions)
  • Vimium for Chrome is a solid example that closed DOMs hurt
• bookmarklets (ie. bookmarks with javascript: URIs)
• in-browser testing frameworks
  • e.g. Selenium WebDriver
• generic libraries (e.g. for jQuery)
• programs using a spec.-compliant DOM API outside of a browser.
  • PhantomJS programs will suffer from this if/when it is updated to use a newer WebKit
  • Programs using a node.js library (eg. jsdom) which meets the spec. will struggle similarly
• any other accessibility, testing or automation framework that uses the DOM as its interface to the document.

In this case, it's not trivial to write a code that walks across shadow trees of components written by different teams or third parties.

I thought this was the point of shadow DOM: to make it harder. If your (generic) program needs to support the increasing number of shadow DOM enabled documents, then it seems pretty manageable to me (unless I've missed anything):

• use event.path[0] instead of event.target
• recursively walk down shadow DOMs of activeElement from document, instead of document.activeElement
• any DOM traversal just needs to check for and walk into a shadowRoot when element an element has one, falling back to normal walking through children
• add a listener to blur/focus/etc. events on each shadowRoot in event.path (to capture changes of focus within a shadow DOM)

any JS code that relies on certain DOM structure in that component's shadow DOM would break.

I agree that this is undesirable. But any JS code doesn't rely on (or even know about) components and their closed shadow DOMs, but need access to them, are broken by their use in the first place. I don't think the extra (circumventable) safety warrants the (often uncircumventable) breakage for this code.

any generic code that walks across DOM to do some work should be treating each custom element instance like a regular builtin HTML element.

The same we'd deal with video elements, input elements, etc... which can be modeled as builtin elements with UA shadow roots (WebKit and Blink DO use shadow roots to implement these elements).

We know what to do with regular builtin HTML elements. The only thing we can do with custom elements we don't know about is ignore them. This is a failure of the DOM.

[dylanb]

@mrmr1993 I can definitely see how Vimium would need access to all shadow DOM - including standard elements. It would be easier for programs that don't require access to these internals to explicitly exclude them like axe-core is planning to do for the <marquee> element - which appears with an open shadow DOM in Chrome.

[rniwa]

• browser extensions (specifically WebExtensions)
  • Vimium for Chrome is a solid example that closed DOMs hurt

What's exposed to browser extensions is up to each UA vendor. So each browser vendor can totally expose a new API that forces all shadow roots to be open, for example.

• bookmarklets (ie. bookmarks with javascript: URIs)
• in-browser testing frameworks
  • e.g. Selenium WebDriver

I could imagine that scripts injected by WebDriver should have access to closed shadow roots. That work could be spec'ed in WebDriver.

• generic libraries (e.g. for jQuery)

Why would generic library need to access things inside a closed shadow tree?

• programs using a spec.-compliant DOM API outside of a browser.
  • PhantomJS programs will suffer from this if/when it is updated to use a newer WebKit
  • Programs using a node.js library (eg. jsdom) which meets the spec. will struggle similarly

How could closed shadow trees cause problems for PhantomJS? Why are nodes.js and PhantomJS special with regards to closed shadow trees?

• any other accessibility, testing or automation framework that uses the DOM as its interface to the document.

Saying that some libraries and frameworks may have an issue because they need to access nodes in a closed shadow tree is a bit tautological. We need a refutable statement. Otherwise, we can keep going around in circles based on each person's opinion.

[dylanb]

@rniwa how many examples do you want before you admit that your a priori opinion is possibly not correct?

What is a valid argument in your opinion?

Is the Vimium example not obviously a valid example of an application? What about that application makes you think it does not require access to the shadow DOM?

Here is another example, albeit forward-looking: the Web standardization process has accepted polyfilling future functionality as a way to prove out the validity/value and appropriateness of a new extension. It is impossible to polyfill anything that would need access to all shadow DOMs if they are closed. This cannot be circumvented with user agent permissions.

[matthewp]

I think he did address the Vimium example; browser extensions are non-standard, the browser can allow them to access closed Shadow DOM if they want to. It's a bug to report to Chrome's extension API.

Personally I do not use closed shadow roots because I just don't care if outside code accesses it, but I can see the argument for it just fine. Let me ask you this @dylanb, does it cause problems for you that you can't access built-in element's shadow DOM? If not, then why is it a problem for custom elements?

[dylanb]

Example of a polyfill which will not be possible:

Implementing a CSS4 :has() polyfill will not work in all the situations, in particular in combination with the >>> combinator.

[rniwa]

@rniwa how many examples do you want before you admit that your a priori opinion is possibly not correct?

The number of example doesn't matter as much as the content of examples.

What is a valid argument in your opinion?

There must be a use case that can't be addressed in the presence of any closed shadow trees, which outweighs all the benefits closed shadow trees provide. But the existence of such a use case seems very unlikely at this point because:

1. We always have an option not to use shadow trees
2. We've had 3-4 years of continuous discussion on this matter and nobody has come up with such an example.

But I welcome anyone to prove me wrong.

Is the Vimium example not obviously a valid example of an application? What about that application makes you think it does not require access to the shadow DOM?

No, because it's a browser extension. Each UA can expose whatever API to make closed shadow trees accessible to browser extensions. In fact, I've added a special API internal to Safari that enables such a feature should Safari team decide to add such a capability to its extension API. In general, browser extensions exist outside the realm of DOM and HTML standards; we don't encourage or discourage the existence of any feature or lack thereof in that space.

Here is another example, albeit forward-looking: the Web standardization process has accepted polyfilling future functionality as a way to prove out the validity/value and appropriateness of a new extension. It is impossible to polyfill anything that would need access to all shadow DOMs if they are closed. This cannot be circumvented with user agent permissions.

Example of a polyfill which will not be possible:

Implementing a CSS4 :has() polyfill will not work in all the situations, in particular in combination with the >>> combinator.

Why would a web component needs to be able to use the polyfill included outside the shadow tree? Again, if this is the first party or collaborating component, then such a component can use either a convention to expose its shadow tree (e.g. getShadowRoot() method) or call some function in the polyfill to enable it in the shadow tree.

If the component was written by a third party or is otherwise non-collaborating, then such a component is unlikely to be expecting the existence of such a polyfill. In fact, the presence of such a polyfill may break or have unintended consequences on the component depending on what the polyfill is doing. So I'd argue that the fact a polyfill doesn't spill into each component's shadow tree is a feature, not a bug, of shadow DOM.

[dylanb]

Are you saying that a polyfill for the CSS4 selectors is not a valid/useful thing to implement?

Are you saying that an author of a document who wants to use a CSS4 selector in combination with the >>> combinator is not a valid use case of a CSS4 selector?

[dylanb]

Here is a product that cannot work properly on a site that uses closed shadow DOM components that are not also standard HTML elements https://sitecues.com/

[rniwa]

Are you saying that a polyfill for the CSS4 selectors is not a valid/useful thing to implement?

That's not at all I'm saying. I'm pointing out that a polyfill included outside the shadow tree not spilling into a component's shadow tree is a feature, not a bug if the component was written by someone else. It avoids breaking that component even if it wasn't written to work well with the newly polyfill.

Here is a product that cannot work properly on a site that uses closed shadow DOM components that are not also standard HTML elements https://sitecues.com/

So it seems that Sitecues have two products. One for users which adds extra accessibility features to the browser, and another which adds those features on a website if the author wishes to have it.

The first product exists outside the realm of the standardization process since it's basically a browser plugin / extension. For the second product, either each component can expose extra information to the library using some API, website can annotate each component with whatever information this tool needs (this is probably the most preferable approach since we can add new builtin HTML elements, and there should be a mechanism to make it compatible with this tool), or each website can include a script synchronously at the top which overrides Element.prototype.attachShadow to provide some hooks.

But really, a well written component shouldn't need any extra work to make it possible to adjust the font size since each component should be specifying its font size using rem, em, or %.

Dictation is a bit hard to implement in a website without a component or a website cooperating for sure but then if a website contains an cross-origin iframe, it won't work across iframe boundaries so I'm not sure if adding dictation feature to a component written by a third party which contains semantically important content in its shadow tree is really a common use case.

In cases where a component is written by a third party, a tab view for example, the content semantically important to the user would be included as children of its shadow host so dictation would just work. In cases where a component is an integral part of the website, e.g. app container is a component itself, then that component can just expose whatever needed for dictation to work.

[dylanb]

ok, so we have two good examples then of valid applications that cannot be implemented with closed shadow DOM:

1. Sitecues dictation
2. polyfills

[rniwa]

I wouldn't say either example is "good".

The dictation can be implemented by either making websites or components collaborate. Furthermore, adding the dictation to a website that's not otherwise designed to do so is usually done at the browser level as a plugin / extension. If a website truly needed a dictation, e.g. e-book site, then that website can natively support & implement dictation as needed everywhere.

For polyfils, I don't think polyfills spilling into a component's shadow tree by default is a good idea at all because polyfills can break components that don't expect them to exist. These kind of isolations and opt-ins are precisely what shadow DOM API should provide.

[rniwa]

I'd note that we didn't spend a week or two thinking about these problems and come to the conclusion that the closed shadow tree is the way to go. We spent 2+ years continuously discussing amongst experienced software engineers who had been working on the Web technology over decades to reach our consensus. And this is precisely why we reached the consensus to keep mode as the mandatory option during W3C F2F meeting: because this topic is extremely contentious and either party arguing for open or closed shadow tree couldn't convince the other party.

It would take some serious thinking and groundbreaking argument or discovery on your part to convince us otherwise.

[rniwa]

Another thing. The canvas element might be hostile to Sitecues's dictation feature because any text drawn in the canvas can't be read by Sitecues unless it implements OCR of some sort. But we didn't remove the canvas element. We instead added enhancements to the canvas to make it more accessible so that tools like Sitecues and browser's native assertive technology could get necessary information out of a canvas element.

Similarly, if there is a specific scenario in which a closed shadow trees would break use cases, then we can make improvements to custom elements or shadow DOM API to address those use cases instead of letting them explore contents inside shadow trees manually. This approach has an added benefit of letting each component decide what's visible to tools like Sitecues.

[dylanb]

@rniwa I have a different proposal: how about user agents treat the built-in standard HTML elements in a special way (by closing their shadow DOM) and make all other shadow DOM open.

This would meet the original requirements of having closed shadow DOM for UA elements but solve all the problems that you are explaining away by (and I am paraphrasing your posts) essentially telling people either not to use shadow DOM, not to use closed shadow DOM (collaborate) or simply not to implement their application the way they would like to.

[mrmr1993]

What's exposed to browser extensions is up to each UA vendor. So each browser vendor can totally expose a new API that forces all shadow roots to be open, for example.

@rniwa If only there was a place to standardise such an API, or some action we could take to make the API unnecessary, to help reduce the burden for developers... It's also worth noting that none of the browsers (Chrome, Firefox, Edge) supporting Web Extensions have previously needed to expose any supplemental APIs for DOM access; the DOM has been generic, well-specified and powerful enough to read and modify the document in all the necessary ways.

We need a refutable statement.

I'll give you ~four~(edit:) five:

• With closed shadow roots, the DOM is no longer generic, well-specified and powerful enough to read and modify the document in the ways the page's scripts can as a whole.
• It is a waste of implementers' time and effort to each discover the limited access and develop a non-standard workaround.
• It is a waste of developers' time and effort to each discover the limited access, request a non-standard workaround and/or work out how it should be used.
• The protection that the component developer is given from the document's author are illusory.
• (edit:) Closed shadow DOMs let any developer create arbitrary self-contained, undocumented extensions to the DOM. The average developer cannot be relied upon to do this usefully, compatibly or well.

How could closed shadow trees cause problems for PhantomJS? Why are nodes.js and PhantomJS special with regards to closed shadow trees?

Scripts that use PhantomJS to interact with documents use the DOM to do so. Some developers will need to be able to read/manipulate content in shadow DOMs programmatically to make their applications work correctly with pages using them. They cannot do this without PhantomJS and node.js adding their own non-standard workarounds.

Why would generic library need to access things inside a closed shadow tree?

Suppose a library wanted to provide an easy way to attach keyboard shortcuts to a page, except when the user is entering text into an input (<input>, <textarea> <* contenteditable="true">, etc.). This library will function incorrectly (ie. interpret input-generating keypresses as keyboard shortcuts) for any pages containing an input inside a closed shadow DOM.

(Full disclosure: this is what Vimium does, but browser-wide instead of for a single document.)

[matthewp]

I disagree completely @mrmr1993, closed shadow DOMs just give developers the same abilities that built-in elements have. If a new built-in form element were created then Vimium wouldn't work with it either. The solution here is to define an interface to interact with form elements that is generic enough that Vimium and others don't need to special case for every element type. These are the events to listen to changes and the properties to get/set the element's underlying value. There's no reason why if my custom element (with a closed shadow) emitted those events and had the correct properties why Vimium wouldn't just work with those.

[mrmr1993]

does it cause problems for you that you can't access built-in element's shadow DOM? If not, then why is it a problem for custom elements

@matthewp built-in elements' behaviours and properties are comprehensive, well-defined, and easy-to-find. There are occasional limitations (e.g. setting the cursor position in an <input type="date" />), but nearly everything you might want to do with them is possible.

By contrast, custom elements need not have any useful/relevant behaviours and properties. Even when they do, code that isn't written for them has no reliable way of using them, since it can't "read the documentation".

[mrmr1993]

There's no reason why if my custom element (with a closed shadow) emitted those events and had the correct properties why Vimium wouldn't just work with those.

@matthewp Of course there isn't. But Vimium (or a library doing the same) is unreliable if it depends on every developer (using a shadow DOM) on every page correctly and consistently implementing the same features. This makes for awful UX.

[mrmr1993]

To put it another way: open shadow DOMs give a (in fact the only) reliable standard API for non-standard components; closed shadow DOMs give you nothing.

[rniwa]

It's also worth noting that none of the browsers (Chrome, Firefox, Edge) supporting Web Extensions have previously needed to expose any supplemental APIs for DOM access; the DOM has been generic, well-specified and powerful enough to read and modify the document in all the necessary ways.

That's absolutely untrue. Browsers already have to provide a mechanism to inject code into every cross-origin iframe's document even though the top-level document ordinarily don't have access to.

[mrmr1993]

the DOM has been generic, well-specified and powerful enough to read and modify the document in all the necessary ways.

That's absolutely untrue. Browsers already have to provide a mechanism to inject code into every cross-origin iframe's document even though the top-level document ordinarily don't have access to.

I think you're being a bit disingenuous. That's not an API to read or modify the document in any way. It does create an execution context with access to read or modify the document, but this is still solely via the DOM.

[rniwa]

We need a refutable statement.

I'll give you ~four~(edit:) five:

• With closed shadow roots, the DOM is no longer generic, well-specified and powerful enough to read and modify the document in the ways the page's scripts can as a whole.

This is demonstratively false. You can simply override Element.prototype.attachShadow to do the same. But more importantly, the whole point of shadow DOM is to provide encapsulation. Encapsulation by definition hides information of each component from the rest. This is a feature, not a bug.

• It is a waste of implementers' time and effort to each discover the limited access and develop a non-standard workaround.

Why is not being able to access a shadow tree's content considered a limitation and something that requires a workaround? This isn't really a refutable but rather an opinion.

• It is a waste of developers' time and effort to each discover the limited access, request a non-standard workaround and/or work out how it should be used.

Ditto.

• The protection that the component developer is given from the document's author are illusory.

Illusory as in they can work around by overriding Element.prototype.attachShadow, or any other method in the same global? That is the limitation of the ECMAScript, not DOM. If ECMAScript provided a way to get a separate realm (and its own global object), or had a reliable way to get the original builtin functions, the protection would not be illusory.

• (edit:) Closed shadow DOMs let any developer create arbitrary self-contained, undocumented extensions to the DOM. The average developer cannot be relied upon to do this usefully, compatibly or well.

This seems like an argument against any average developer writing any component at all. It doesn't matter whether a component uses a shadow tree or not since any component can be ill-formed. FWIW, a component can be written using a cross-origin iframe or a canvas element in which case nobody else wouldn't have access to its content even today.

[mrmr1993]

It is a waste of implementers' time and effort to each discover the limited access and develop a non-standard workaround.

Why is not being able to access a shadow tree's content considered a limitation and something that requires a workaround? This isn't really a refutable but rather an opinion.

If useful programs can't be written to function as intended, that seems like a limitation. Especially if they could be before. Is it not a limitation that a developer can't write a keyboard shortcut library of the kind I described as above?

• The protection that the component developer is given from the document's author are illusory.

Illusory as in they can work around by overriding Element.prototype.attachShadow, or any other method in the same global? That is the limitation of the ECMAScript, not DOM. If ECMAScript provided a way to get a separate realm (and its own global object), or had a reliable way to get the original builtin functions, the protection would not be illusory.

This is the whole point of the closed shadow DOM though, no? Either the DOM is important for multiple contexts, in which case they may not have document awareness and need shadow DOM access to behave correctly with custom elements, or they're not, in which case they're embedded in the document, use ECMAScript and the protection is illusory.

[rniwa]

It is a waste of implementers' time and effort to each discover the limited access and develop a non-standard workaround.

Why is not being able to access a shadow tree's content considered a limitation and something that requires a workaround? This isn't really a refutable but rather an opinion.

If useful programs can't be written to function as intended, that seems like a limitation. Especially if they could be before. Is it not a limitation that a developer can't write a keyboard shortcut library of the kind I described as above?

Would you argue that private variables in languages like C++ and Java are limitations that prevent useful programs to be written because other code can't access those states and implement useful programs?

Or would you argue that the canvas is a limitation because it won't allow the same library to provide keyboard shortcuts into text drawn in the canvas?

• The protection that the component developer is given from the document's author are illusory.

Illusory as in they can work around by overriding Element.prototype.attachShadow, or any other method in the same global? That is the limitation of the ECMAScript, not DOM. If ECMAScript provided a way to get a separate realm (and its own global object), or had a reliable way to get the original builtin functions, the protection would not be illusory.

This is the whole point of the closed shadow DOM though, no? Either the DOM is important for multiple contexts, in which case they may not have document awareness and need shadow DOM access to behave correctly with custom elements, or they're not, in which case they're embedded in the document, use ECMAScript and the protection is illusory.

There is a discussion for private states in ECMAScript on track, and we've been discussing about the fully isolated components for years now.

[mrmr1993]

Would you argue that private variables in languages like C++ and Java are limitations that prevent useful programs to be written because other code can't access those states and implement useful programs?

I would argue that a user can't interactively render a collection of private variables, nor pass it to a program, expecting it to act upon their mental model of that rendering.

Or would you argue that the canvas is a limitation because it won't allow the same library to provide keyboard shortcuts into text drawn in the canvas?

It's an image. At a facile level, if you're not using document objects to render your content, then you have nothing to do with the DOM, and the DOM has nothing to do with you. This sucks for all planned or unplanned external interactivity, but it shouldn't be a consideration for this discussion.

There is a discussion for private states in ECMAScript on track, and we've been discussing about the fully isolated components for years now.

I think the same point as for C++ and Java applies. It's not a 'thing' as far is the user is concerned. Go wild.

[rniwa]

There is a discussion for private states in ECMAScript on track, and we've been discussing about the fully isolated components for years now.

I think the same point as for C++ and Java applies. It's not a 'thing' as far is the user is concerned. Go wild.

Well, then that same argument holds for closed shadow trees. It's not really a thing to go into some component's shadow tree and mess with it.

[mrmr1993]

I think the same point as for C++ and Java applies. It's not a 'thing' as far is the user is concerned. Go wild.

Well, then that same argument holds for closed shadow trees.

Hardly. The DOM is always meaningful, since it represents the user interface/user-facing content in a standardised way. If you give me a DOM (and the relevant styles, admittedly), I can tell you what it represents and do useful things with it. Not so with arbitrary internal private states -- their representation is only meaningful in the context of that specific program.

It's not really a thing to go into some component's shadow tree and mess with it.

I think you may have misinterpreted me: I meant that ECMAScript internal state isn't a virtual object in the mind of the user. DOM elements certainly are. Nonetheless, this is patently false; programs already do this. That you don't like it doesn't make it not a 'thing'.

The thing that's made the web so powerful up until now is the DOM. Search engines rely on the DOM. Libraries rely on the DOM. Extensions rely on the DOM. Accessibility programs rely on the DOM. People have been able to re-use huge amounts of code because the underlying objects are all the same, and all accessible.

This all dies when your documents are full of opaque <main-content>s and <autocomplete-input>s and mine are full of <article-body>s and <filter-dropdown>s. Or worse, 'empty' <div>s. The objects in the DOM can't reliably convey any meaning to cross-document programs, unless they learn to read the developers' minds. This ruins the web.

I'll quote my example from above here, in the hopes that you have a response to it:

Suppose a library wanted to provide an easy way to attach keyboard shortcuts to a page, except when the user is entering text into an input (<input>, <textarea> <* contenteditable="true">, etc.). This library will function incorrectly (ie. interpret input-generating keypresses as keyboard shortcuts) for any pages containing an input inside a closed shadow DOM.

[BronislavKlucka]

Hi,

you are missing the point, /closed/ mode is not wrongly introduced because you cannot mess with the internals, it was created precisely for this purpose: so you cannot mess with the internals. You are looking at this issue with very narrow perspective of very few use cases, completely ignoring the pain of web apps developers they have been dealing with for decades: the global DOM space, where everything could affect everything. The common way, how desktop apps are developed is that you have/buy RAD with e.g. 20 components and through out the years you write/purchase/download hundreds of others and the beauty is, that those are self contained black boxes that always works together regardless of any kind of combination of any number of component from any number of developers (exceptions being possible class naming collisions and unique resources access). The second beauty of this simply is the fact, that developers who are using those components are informed "you can touch this, this is fixed and you cannot rely on that, that is internal and can change with every minor release". If you chose to somehow rely on internals and it breaks, it's your fault, not authors.

If you ever tried to combine even 2 web/js component libraries together? pain... and then upgrade one... Yes /closed/ mode is something new and strange but something web platforms needs to become fully fledged application run time (yes, we could somehow do this before... you can walk to Rome, but I assume air plain or car is much better solution).

You are arguing that " This all dies when your documents are full of opaque ||s and ||s and mine are full of ||s and ||s. Or worse, 'empty' |

|s. " but you are effectively describing the issue you/we had for decades: "opaque 'empty' |

|s", applications build by divs and spans, filled by another ||divs and spans generated via XMLHTTPRequest... same diff for you (no officially described semantics what so ever), huge difference for those who write/maintain such projects.

Instead of refusing several years of discussions, the way to solve this is come up with set of attributes that would be useful to expose/describe, the same way WAI-ARIA works to describe authored controls. Do you want to be informed about the fact that control is editable? " Suppose a library wanted to provide an easy way to attach keyboard shortcuts to a page, except when the user is entering text into an input (||, || |<* contenteditable="true">|, etc.). This library will function incorrectly (ie. interpret input-generating keypresses as keyboard shortcuts) for any pages containing an input inside a closed shadow DOM. " let's introduce editable attribute, that would be default on controls mentioned above (unless readonly/disabled) and can be used on other controls. This is the way how it worked before (e.g. input extensions) and how it could work again. If you have experience with this, if you have some numbers of those attributes in mind, offer them.</p> <p>Solution we have now is not perfect, but we should work on it, not just simply say "it has few issues, so forget about it all"</p> <p>-- </p> <pre><code> Bronislav Klučka</code></pre> </div> </div> <div class="comment"> <div class="user"> <a rel="noreferrer nofollow" target="_blank" href="https://github.com/mrmr1993"><img src="https://avatars.githubusercontent.com/u/1847343?v=4" />mrmr1993</a> commented <strong> 7 years ago</strong> </div> <div class="markdown-body"> <blockquote> <p>you are missing the point, /closed/ mode is not wrongly introduced because you cannot mess with the internals, it was created precisely for this purpose: so you cannot mess with the internals.</p> </blockquote> <p>I don't think I am. I'm trying to argue that this change is harmful, and that the justifications are not proportionate to the harms.</p> <blockquote> <p>You are looking at this issue with very narrow perspective of very few use cases, completely ignoring the pain of web apps developers they have been dealing with for decades: the global DOM space, where everything could affect everything. The common way, how desktop apps are developed is that you have/buy RAD with e.g. 20 components and through out the years you write/purchase/download hundreds of others and the beauty is, that those are self contained black boxes that always works together regardless of any kind of combination of any number of component from any number of developers (exceptions being possible class naming collisions and unique resources access). The second beauty of this simply is the fact, that developers who are using those components are informed "you can touch this, this is fixed and you cannot rely on that, that is internal and can change with every minor release". If you chose to somehow rely on internals and it breaks, it's your fault, not authors.</p> </blockquote> <p>We're all in agreement on this. Notice how nobody is arguing against open shadow DOMs, which satisfy all of these things.</p> <blockquote> <p>If you ever tried to combine even 2 web/js component libraries together? pain... and then upgrade one...</p> </blockquote> <p>Agreed, it's nasty. Open shadow DOMs are a great way to help improve this pain point.</p> <blockquote> <p>Yes /closed/ mode is something new and strange but something web platforms needs to become fully fledged application run time</p> </blockquote> <p>Closed mode isn't new and strange: restricting access to things isn't that revolutionary a concept, and it being new doesn't make it a good idea. Perhaps shadow DOMs are new and strange, but they are a brilliant idea, something the web needs, and in no way require closed mode.</p> <blockquote> <p>you are effectively describing the issue you/we had for decades: "opaque 'empty' |<div>|s", applications build by divs and spans, filled by another ||divs and spans generated via XMLHTTPRequest... same diff for you (no officially described semantics what so ever), huge difference for those who write/maintain such projects.</p> </blockquote> <p>It's actually a world of difference for me. The officially described semantics <em>were the DOM</em>. Anything you wanted or needed to know about the user-facing content could be found by looking in the DOM. Not so for closed shadow roots: you may as well replace every element with a closed shadow DOM with a <code><useless-element></code>.</p> <p>The huge difference for writers/maintainers comes from custom elements, which can function perfectly well with an open-only shadow DOM.</p> <blockquote> <p>Instead of refusing several years of discussions, the way to solve this is come up with set of attributes that would be useful to expose/describe, the same way WAI-ARIA works to describe authored controls. Do you want to be informed about the fact that control is editable?</p> </blockquote> <p>I <em>still</em> can't find anything in those several years of discussion that persuades me that closed shadow roots are a net positive.</p> <p>The set of attributes that would be useful to expose/describe is the DOM. Or is there something in the DOM API that you consider not useful to have exposed?</p> <blockquote> <p>let's introduce editable attribute, that would be default on controls mentioned above (unless readonly/disabled) and can be used on other controls.</p> </blockquote> <p>Let's say your shadow DOM contains a text input and a checkbox. Is it editable? If not, the behaviour is obviously broken in the input box; otherwise, behaviour is incorrect when the checkbox has focus. This is an example of a family of non-trivial problems, and the glaringly obvious solution is to have some API to expose the internal DOM.</p> <blockquote> <p>Solution we have now is not perfect, but we should work on it, not just simply say "it has few issues, so forget about it all"</p> </blockquote> <p>It's a tack-on to open shadow roots, not a solution in its own right. Exposing the shadow DOM is a detail of the larger shadow roots concept, and none of the other very significant benefits of shadow roots disappear if we "forget all about it".</p> </div> </div> <div class="comment"> <div class="user"> <a rel="noreferrer nofollow" target="_blank" href="https://github.com/mrmr1993"><img src="https://avatars.githubusercontent.com/u/1847343?v=4" />mrmr1993</a> commented <strong> 7 years ago</strong> </div> <div class="markdown-body"> <p>I worry that I might have missed something fundamental here. Can anybody clarify for me:</p> <ul> <li>Do closed shadow roots provide any use, or make anything possible, that open ones do not?</li> <li>Who benefits from closed shadow DOMs? Who suffers from them? Is the trade-off worth it?</li> </ul> <p>As far as I can tell, the answers are:</p> <ul> <li>No, except possibly fulfilling a sense of "getting the damn kids off my lawn".</li> <li>Component developers 'benefit' by taking and withholding control. Developers 'benefit' from being unable to do some things 'for their own good'. This looks like a restriction rather than a feature, based off weak reasoning, and doesn't feel worth it to me.</li> </ul> <p>(Please read "getting the damn kids off my lawn" as "knowing better than and what's best for other developers, and wanting to impose it", if the former offends your sensibilities.)</p> </div> </div> <div class="comment"> <div class="user"> <a rel="noreferrer nofollow" target="_blank" href="https://github.com/BronislavKlucka"><img src="https://avatars.githubusercontent.com/u/13207465?v=4" />BronislavKlucka</a> commented <strong> 7 years ago</strong> </div> <div class="markdown-body"> <p>yes It does, the same thing as in any generic desktop app development: the difference what is and what is not to be messed with, the argument is not about "I§m the king of the world and I know best", as it's not in other environments. Clear encapsulation is the point. Open shadow root does not provide that. I do not know your experience ouside web, but I can open Delphi (Object Pascal), Visual Studio (C#+.NET) and thou syntax is very different, in semantics it makes no difference, I can tell what is public, what is private, not because being the best, but to define interface. I cannot do that with open shadow DOM, which part is or is not to be messed with. Who benefits from it? Every single programmer using 3rd party component. Who suffer? very few use cases.</p> <p>The argument may not persuade you personally, but the fact that a lot of people discussing this agreed upon the fact that both are needed my mean something. I can see usecases for open and closed.</p> </div> </div> <div class="comment"> <div class="user"> <a rel="noreferrer nofollow" target="_blank" href="https://github.com/mrmr1993"><img src="https://avatars.githubusercontent.com/u/1847343?v=4" />mrmr1993</a> commented <strong> 7 years ago</strong> </div> <div class="markdown-body"> <blockquote> <p>It does, the same thing as in any generic desktop app development</p> </blockquote> <p>Just because it's done elsewhere doesn't mean it's right or well-justified.</p> <blockquote> <p>the argument is not about "I§m the king of the world and I know best", as it's not in other environments.</p> </blockquote> <p>Indeed, the argument is long resolved elsewhere, since the language implementations have historically implemented it. This does not -- and should not -- preclude us from making a better or different decision here, and considering fully the implications it will have.</p> <blockquote> <p>I can open Delphi (Object Pascal), Visual Studio (C#+.NET) and thou syntax is very different, in semantics it makes no difference, I can tell what is public, what is private, not because being the best, but to define interface. I cannot do that with open shadow DOM, which part is or is not to be messed with.</p> </blockquote> <p>In my experience, this has been a major pain point for developers when dealing with under-specified components. I hope you are not talking personally about being unable to tell. Clearly an open shadow DOM is something separate, and should be obvious by its inconvenience that it is not designed to be used casually or on a whim.</p> <blockquote> <p>Who benefits from it? Every single programmer using 3rd party component.</p> </blockquote> <p>How? I'm still unclear on this. Perhaps you could give me an example of a 3rd party component developer changing an implementation detail from public to private and it tangibly benefiting one of these programmers.</p> <blockquote> <p>I can see usecases for open and closed.</p> </blockquote> <p>Some examples might be helpful. Why would some components require complete closure of implementation details and others not? How would you ensure that the right one is used for the right usecases?</p> </div> </div> <div class="comment"> <div class="user"> <a rel="noreferrer nofollow" target="_blank" href="https://github.com/BronislavKlucka"><img src="https://avatars.githubusercontent.com/u/13207465?v=4" />BronislavKlucka</a> commented <strong> 7 years ago</strong> </div> <div class="markdown-body"> <p>On 14.5.2017 18:37, Matthew Ryan wrote:</p> <blockquote> <pre><code>It does, the same thing as in any generic desktop app development</code></pre> <p>Just because it's done elsewhere doesn't mean it's right or well-justified.</p> <p>correct, but I'll make the point below discussing documentation</p> <pre><code>the argument is not about "I§m the king of the world and I know best", as it's not in other environments.</code></pre> <p>Indeed, the argument is long resolved elsewhere, since the language implementations have historically implemented it. This does not -- and should not -- preclude us from making a better or different decision here, and considering fully the implications it will have.</p> <p>But it was discussed, for several years. And a lot of implementation details have been considered, there have always been opposition against /closed/ mode and their arguments were heard, but the decision have to be made and have been made. The argument is the other way around: we have the experience, it works, it works exactly as it should and it's pretty much always stressed to use encapsulation</p> <pre><code>I can open Delphi (Object Pascal), Visual Studio (C#+.NET) and thou syntax is very different, in semantics it makes no difference, I can tell what is public, what is private, not because being the best, but to define interface. I cannot do that with open shadow DOM, which part is or is not to be messed with.</code></pre> <p>In my experience, this has been a major pain point for developers when dealing with under-specified components. I hope you are not talking personally about being unable to tell. Clearly an open shadow DOM is something separate, and should be obvious by its inconvenience that it is not designed to be used casually or on a whim.</p> <p>Of course the decision on /open///closed /should not be on the whim, even usage of shadow dom of any kind should not be, it should not be used just because we have it. The documentation is nice thing, but the best documentation is always the code itself, why should I look on object, listing properties/fields/methods and the look into documentation to see which is public which is not... Why should anyone spent time writing/reading documentation on what is private/protected/public? this does not even exists in DOM itself, you either can see it on object = you can mess with that, or you do not see it at all.</p> <pre><code>Who benefits from it? Every single programmer using 3rd party component.</code></pre> <p>How? I'm still unclear on this. Perhaps you could give me an example of a 3rd party component developer changing an implementation detail from public to private and it tangibly benefiting one of these programmers.</p> <p>This is not about changing, this is about getting the component and knowing immediately, it is about knowing, that it's pretty much always save to upgrade, knowing, that public is safe to mess around with, knowing that you can rely on that. Again, trying to integrate 2 JS/DOM framework is pain. Everywhere else it's seamless.</p> <pre><code>I can see usecases for open and closed.</code></pre> <p>Some examples might be helpful. Why would some components require complete closure of implementation details and others not?</p> <p>I have always been proponent of /closed/ only, so I'll have to live with the fact, that sometimes I'll look at some component and be pretty much where I'm today, knowing nothing :). For me, talking about components, there should always be /closed/ mode: "here is an interface for this component, that's it", as it has been proven to work for decades everywhere else and with the DOM native elements and native JS objects. There was never any access to all in web platform. It may be convenient to have only some native components. But it's limiting. /Open/ mode for me? Modules. I can see e.g. putting together a module for files (list control to display, buttons to add, delete, pagination) and use it in different places in project (files for customers, files for tasks, files for projects). It's self contained, as shadow dom, to be able to be placed anywhere without breaking the module itself, or module breaking surroundings, but it can freely be used. And list control is, of course (for me) closed.</p> <p>How would you ensure that the right one is used for the right usecases?</p> <p>Pretty much as with anything you code?</p> </blockquote> <p>I get that all I write about can mean nothing to you, but again, there have been a loooong discussion already.</p> <pre><code> Bronislav Klučka</code></pre> </div> </div> <div class="comment"> <div class="user"> <a rel="noreferrer nofollow" target="_blank" href="https://github.com/mrmr1993"><img src="https://avatars.githubusercontent.com/u/1847343?v=4" />mrmr1993</a> commented <strong> 7 years ago</strong> </div> <div class="markdown-body"> <blockquote> <p>But it was discussed, for several years. And a lot of implementation details have been considered, there have always been opposition against /closed/ mode and their arguments were heard, but the decision have to be made and have been made. The argument is the other way around: we have the experience, it works, it works exactly as it should and it's pretty much always stressed to use encapsulation</p> </blockquote> <p>This <em>still</em> doesn't mean that the right decision has been made, either in this spec. or by other language implementers. In general, experience with a bad decision is still experience with a bad decision. Unfortunately I'm not at all persuaded that this isn't one.</p> <blockquote> <p>Of course the decision on /open///closed /should not be on the whim, even usage of shadow dom of any kind should not be, it should not be used just because we have it.</p> </blockquote> <p>I only meant it shouldn't be used on a whim, only when it is useful and unavoidable.</p> <blockquote> <p>This is not about changing, this is about getting the component and knowing immediately, it is about knowing, that it's pretty much always save to upgrade, knowing, that public is safe to mess around with, knowing that you can rely on that.</p> </blockquote> <p>I know it's not about changing: I was hoping for an example of a tangible benefit, and thought that might be a good way to demonstrate one. Still, if you aren't using code that touches the shadow DOM of a component, I can't see why you wouldn't be safe to upgrade it regardless.</p> <hr /> <p>If the closed shadow DOM is supposed to give the <em>document author</em> these assurances, why is it that the <em>component developer</em> is the one getting the say over open vs. closed? The document author is at the receiving end of the safety vs. power trade-off, so surely they should have control over the decision.</p> </div> </div> <div class="comment"> <div class="user"> <a rel="noreferrer nofollow" target="_blank" href="https://github.com/rniwa"><img src="https://avatars.githubusercontent.com/u/285965?v=4" />rniwa</a> commented <strong> 7 years ago</strong> </div> <div class="markdown-body"> <blockquote> <blockquote> <blockquote> <p>I think the same point as for C++ and Java applies. It's not a 'thing' as far is the user is concerned. Go wild.</p> </blockquote> <p>Well, then that same argument holds for closed shadow trees.</p> </blockquote> <p>Hardly. The DOM is always meaningful, since it represents the user interface/user-facing content in a standardised way. If you give me a DOM (and the relevant styles, admittedly), I can tell you what it represents and do useful things with it. Not so with arbitrary internal private states -- their representation is only meaningful in the context of that specific program.</p> </blockquote> <p>That is patently false.</p> <blockquote> <p>If the closed shadow DOM is supposed to give the document author these assurances, why is it that the component developer is the one getting the say over open vs. closed? The document author is at the receiving end of the safety vs. power trade-off, so surely they should have control over the decision.</p> </blockquote> <p>Web components provide powers to both library/component authors as well as page authors. The primary benefits of closed shadow trees is the "guarantee" or a strong implication that the components are free to change its implementation without having to worry about its details. This is extremely important for a popular component intended to be upgraded and maintained for a long period of time.</p> <p>We have quite a bit of experience dealing with such a problem in UIKit and AppKit here at Apple. We frequently encounter binary compatibility issues whereby which a third party app was relying on the implementation details of UIKit, AppKit, and other system frameworks. They are not supposed to do that (clearly stated in the policy and what not) but in the practice, many apps do because they can. In those cases, we must either can't make a change (e.g. maintaining same view / CA layer hierarchy), or break the app and notify the authors to have them fix them before those changes take effect. With AppStore and once a year release cycle, this is all possible. However, in the world of the Web, where developers would like to move faster and innovate in a rapid pace, and there is no repository of all Web apps which use a given framework or a library, this will become an impossible issue to resolve.</p> <p>Just as another example, Microsoft once upon a time (not sure if they still do) decided to make it possible for any given Windows system to have multiple versions of the same DLL so that each app that links against a particular version of DLL will continue to work to combat the infamous <a href="https://en.wikipedia.org/wiki/DLL_Hell">DLL hell</a>.</p> </div> </div> <div class="comment"> <div class="user"> <a rel="noreferrer nofollow" target="_blank" href="https://github.com/rniwa"><img src="https://avatars.githubusercontent.com/u/285965?v=4" />rniwa</a> commented <strong> 7 years ago</strong> </div> <div class="markdown-body"> <blockquote> <blockquote> <blockquote> <p>I think the same point as for C++ and Java applies. It's not a 'thing' as far is the user is concerned. Go wild.</p> </blockquote> <p>Well, then that same argument holds for closed shadow trees.</p> </blockquote> <p>Hardly. The DOM is always meaningful, since it represents the user interface/user-facing content in a standardised way. If you give me a DOM (and the relevant styles, admittedly), I can tell you what it represents and do useful things with it. Not so with arbitrary internal private states -- their representation is only meaningful in the context of that specific program.</p> </blockquote> <p>That isn't quite true. A li can be used to represent either an item in a shooping list, a file in the list of uploaded file, etc... The semantics of each element and content depends heavily on the context in which it exists.</p> <blockquote> <blockquote> <p>It's not really a thing to go into some component's shadow tree and mess with it.</p> </blockquote> <p>I think you may have misinterpreted me: I meant that ECMAScript internal state isn't a virtual object in the mind of the user. DOM elements certainly are. Nonetheless, this is patently false; programs already do this. That you don't like it doesn't make it not a 'thing'. </p> </blockquote> <p>Well, not really. In the presence of closed shadow trees, those scripts don't / can't do this. Also, just because it's currently done, it's a good idea.</p> <p>For example, we used to write code with goto and label everywhere before higher-level languages were invented. Do you think a language which hides goto is harmful because it disallows certain use cases?</p> <blockquote> <p>The thing that's made the web so powerful up until now <em>is</em> the DOM. Search engines rely on the DOM. Libraries rely on the DOM. Extensions rely on the DOM. Accessibility programs rely on the DOM. People have been able to re-use huge amounts of code because the underlying objects are all the same, and all accessible.</p> <p>This all dies when your documents are full of opaque <code><main-content></code>s and <code><autocomplete-input></code>s and mine are full of <code><article-body></code>s and <code><filter-dropdown></code>s. Or worse, 'empty' <code><div></code>s. The objects in the DOM can't reliably convey any meaning to cross-document programs, unless they learn to read the developers' minds. This ruins the web.</p> </blockquote> <p>We already live in that world. People are writing apps with a bunch of divs and spans, and that's completely orthogonal to the concept of closed shadow trees.</p> <blockquote> <p>I'll quote my example from above here, in the hopes that you have a response to it:</p> <blockquote> <p>Suppose a library wanted to provide an easy way to attach keyboard shortcuts to a page, except when the user is entering text into an input (<code><input></code>, <code><textarea></code> <code><* contenteditable="true"></code>, etc.). This library will function incorrectly (ie. interpret input-generating keypresses as keyboard shortcuts) for any pages containing an input inside a closed shadow DOM.</p> </blockquote> </blockquote> <p>What happens when one of the components on a page already implemtns its own keyboard shortcuts? With an open shadow tree or without a shadow tree at all, this library will go in and add new keyboard shortcuts.</p> <p>This is why it's very important for each component to opt-in to whatever polyfill or library each page is importing. The idea of automatically enhancing an arbitrary component written by a third party quickly fails in practice.</p> </div> </div> <div class="comment"> <div class="user"> <a rel="noreferrer nofollow" target="_blank" href="https://github.com/mrmr1993"><img src="https://avatars.githubusercontent.com/u/1847343?v=4" />mrmr1993</a> commented <strong> 7 years ago</strong> </div> <div class="markdown-body"> <blockquote> <p>This is why it's very important for each component to opt-in to whatever polyfill or library each page is importing. The idea of automatically enhancing an arbitrary component written by a third party quickly fails in practice.</p> </blockquote> <p>This seems to be the only point where we're not looping around the same arguments.</p> <p>Why is it very important for each <em>component</em> to do this? Sometimes it will be what the document author intends and wants; other times not. Can we not find a compromise where the document author gets to decide which scripts can and can't access which shadow DOMs? Then, those authors have a clear idea of what might break if/when a component is updated, and what will not.</p> <p>For example, would there be any issues with extending the script tag so that:</p> <ul> <li><code><script ... shadow-dom="true"></code> allows the script it includes (and any execution it spawns, such as <code>eval</code>s, event listeners, etc.) to access shadow DOMs;</li> <li><code><script ... shadow-dom="false"></code> does not give the script any shadow DOM access (under the same conditions); <ul> <li>it should also likely restrict scripts it imports to <code>shadow-dom="false"</code></li> </ul></li> <li>possibly <code><script ... shadow-dom="request"></code>, which eg. dispatches an event on the shadow host on <code>.shadowRoot</code> accesses that scripts with <code>shadow-dom=true</code> can catch (<code>event.relatedTarget</code> perhaps being the script element), denying access using <code>event.preventDefault()</code>? <ul> <li>it should also likely convert scripts it imports from <code>shadow-dom="true"</code> to <code>shadow-dom="request"</code></li> </ul></li> </ul> <p>Adding a <code>with {shadowDom: "true" | "false" | "request"}</code> to the spec. for ECMAScript 6 <code>import</code>s could easily provide the same for those. I also think that a default to <code>false</code> for all scripts would be reasonable in this case, for the reassurances it provides.</p> <p>The major issue I have with closed shadow DOMs is that the decision is made piecemeal, by the developer of each component, and (necessarily) without the complete information of their interactions with each author's document. Giving this control to the author at the script level would seem to give the assurances <em>and</em> flexibility that we want between us.</p> </div> </div> <div class="comment"> <div class="user"> <a rel="noreferrer nofollow" target="_blank" href="https://github.com/rniwa"><img src="https://avatars.githubusercontent.com/u/285965?v=4" />rniwa</a> commented <strong> 7 years ago</strong> </div> <div class="markdown-body"> <blockquote> <p>Why is it very important for each <em>component</em> to do this? Sometimes it will be what the document author intends and wants; other times not. Can we not find a compromise where the document author gets to decide which scripts can and can't access which shadow DOMs? Then, those authors have a clear idea of what might break if/when a component is updated, and what will not.</p> </blockquote> <p>Because otherwise pages would start depending on the implementation details of components. It's possible, for example, for a graphing component to be implemented using SVG in one iteration and canvas in another. If page's code ever depended on the component using SVG, then the page would break when the component's version gets updated and it starts using canvas instead of SVG.</p> <p>Ultimately, page authors have the power not to use a component which doesn't expose its shadow tree, or demand that they provide some API to mess with component's shadow tree. However, it's very important for the authors of each component to understand that doing so results in its shadow tree's structure being dependent by its users.</p> <p>If the authors of a given component doesn't care about this, or is committed to maintain the backwards compatibility, then they can either not use shadow tree at all, or use an open mode shadow tree.</p> <p>Either case, we don't believe we should give the unilateral power to users of components to decide whether it's okay to intrude into components' shadow tree or not.</p> </div> </div> <div class="comment"> <div class="user"> <a rel="noreferrer nofollow" target="_blank" href="https://github.com/mrmr1993"><img src="https://avatars.githubusercontent.com/u/1847343?v=4" />mrmr1993</a> commented <strong> 7 years ago</strong> </div> <div class="markdown-body"> <blockquote> <p>Because otherwise pages would start depending on the implementation details of components.</p> </blockquote> <p>I'm still not horrified by this. Sometimes this is useful and/or necessary.</p> <blockquote> <p>It's possible, for example, for a graphing component to be implemented using SVG in one iteration and canvas in another. If page's code ever depended on the component using SVG, then the page would break when the component's version gets updated and it starts using canvas instead of SVG.</p> </blockquote> <p>If a page breaks, is it not an issue for the page's author, rather than the component developer?</p> <blockquote> <p>However, it's very important for the authors of each component to understand that doing so results in its shadow tree's structure being dependent by its users.</p> </blockquote> <p>If a page depends on its internals, then they've taken that risk that updates will break things. It's their decision and their issue. I'm astounded that this position is controversial.</p> <blockquote> <p>Ultimately, page authors have the power not to use a component which doesn't expose its shadow tree, or demand that they provide some API to mess with component's shadow tree.</p> </blockquote> <p>So you would prefer page authors to beg component developers instead of making the decision themselves?</p> <blockquote> <p>Either case, we don't believe we should give the unilateral power to users of components to decide whether it's okay to intrude into components' shadow tree or not.</p> </blockquote> <p>Why can the component decide to opt them in or out of protections, but the page author themself cannot? The power dynamics of the current situation are really unusual, and don't seem to have any real advantages</p> </div> </div> <div class="comment"> <div class="user"> <a rel="noreferrer nofollow" target="_blank" href="https://github.com/treshugart"><img src="https://avatars.githubusercontent.com/u/136556?v=4" />treshugart</a> commented <strong> 7 years ago</strong> </div> <div class="markdown-body"> <blockquote> <p>The power dynamics of the current situation are really unusual, and don't seem to have any real advantages</p> </blockquote> <p>Especially when you can just:</p> <pre><code class="language-js">const oldAttachShadow = HTMLElement.prototype.attachShadow; HTMLElement.prototype.attachShadow = function () { return oldAttachShadow.call(this, { mode: 'open' }); }</code></pre> </div> </div> <div class="comment"> <div class="user"> <a rel="noreferrer nofollow" target="_blank" href="https://github.com/rniwa"><img src="https://avatars.githubusercontent.com/u/285965?v=4" />rniwa</a> commented <strong> 7 years ago</strong> </div> <div class="markdown-body"> <blockquote> <blockquote> <p>Because otherwise pages would start depending on the implementation details of components.</p> </blockquote> <p>I'm still not horrified by this. Sometimes this is useful and/or necessary.</p> </blockquote> <p>The problem is that we've already seen a <a href="https://github.com/w3c/webcomponents/wiki/Shadow-piercing-Combinators-in-the-Wild">similar problem with the shadow piercing combinator</a>.</p> <blockquote> <blockquote> <p>It's possible, for example, for a graphing component to be implemented using SVG in one iteration and canvas in another. If page's code ever depended on the component using SVG, then the page would break when the component's version gets updated and it starts using canvas instead of SVG.</p> </blockquote> <p>If a page breaks, is it not an issue for the page's author, rather than the component developer?</p> </blockquote> <p>The unfortunate reality is that library and framework authors would have to worry about those dependency. We deal with those issues all over the place in AppKit and UIKit or even WebKit / WebKit2 themselves embedded in other applications, and it imposes serious constrains on what we can and cannot change in our implementations.</p> <p>And there is evidence after evidence that this is an issue throughout the industry as I've already enumerated a few.</p> <blockquote> <blockquote> <p>However, it's very important for the authors of each component to understand that doing so results in its shadow tree's structure being dependent by its users.</p> </blockquote> <p>If a page depends on its internals, then they've taken that risk that updates will break things. It's their decision and their issue. I'm astounded that this position is controversial.</p> </blockquote> <p>The problem is that it quickly becomes library & framework author's problem once enough websites decide to take that risk.</p> <blockquote> <blockquote> <p>Ultimately, page authors have the power not to use a component which doesn't expose its shadow tree, or demand that they provide some API to mess with component's shadow tree.</p> </blockquote> <p>So you would prefer page authors to beg component developers instead of making the decision themselves?</p> </blockquote> <p>Yes. They don't even have to beg. Don't use a badly written component, or a component you don't like. Nobody is forcing you to use a component that you don't like so much that you want to go in & modify its shadow tree.</p> <blockquote> <blockquote> <p>Either case, we don't believe we should give the unilateral power to users of components to decide whether it's okay to intrude into components' shadow tree or not.</p> </blockquote> <p>Why can the component decide to opt them in or out of protections, but the page author themself cannot? The power dynamics of the current situation are really unusual, and don't seem to have any real advantages</p> </blockquote> <p>Because they're the one writing a component? If the license allows, page author can just copy / fork the component and modify its script not to use a shadow tree or make it use an open shadow tree; or better yet, directly make necessary changes desired on a particular website.</p> </div> </div> <div class="comment"> <div class="user"> <a rel="noreferrer nofollow" target="_blank" href="https://github.com/mrmr1993"><img src="https://avatars.githubusercontent.com/u/1847343?v=4" />mrmr1993</a> commented <strong> 7 years ago</strong> </div> <div class="markdown-body"> <blockquote> <p>The problem is that we've already seen a <a href="https://github.com/w3c/webcomponents/wiki/Shadow-piercing-Combinators-in-the-Wild">similar problem with the shadow piercing combinator</a>.</p> </blockquote> <p>I can't necessarily see that these are problems. Allowing a document author to theme a component, or to use a library that does so, seem like completely reasonable use cases. That shadow DOMs prevent the <em>unintentional</em> downward leaking of styles is excellent, but it shouldn't mean that they should be completely unstylable from the outside.</p> <p>(Also, the <code>:part()</code> selector seemed like a nice go-between for component theming: well-specified components can be styled in a non-implementation-dependent way, and under-specified components can still be styled by <code>/deep/</code>.)</p> <blockquote> <blockquote> <p>If a page breaks, is it not an issue for the page's author, rather than the component developer?</p> </blockquote> <p>The unfortunate reality is that library and framework authors would have to worry about those dependency. We deal with those issues all over the place in AppKit and UIKit or even WebKit / WebKit2 themselves embedded in other applications, and it imposes serious constrains on what we can and cannot change in our implementations.</p> </blockquote> <p>I think the comparison with AppKit/UIKit and the WebKits is misleading. Since the AppKit/UIKit components are the lowest-level UI components (and the lowest-level, well-specified, reliable API to these components), the comparison should be with UA's native elements.</p> <p>I'll happily agree with <a href="https://github.com/w3c/webcomponents/issues/640#issuecomment-301070698">@dylanb's statement</a> that UA elements <em>should not</em> expose their shadow DOMs, and by analogy that AppKit/UIKit should not expose their internals, since they are not described in a well-specified, reliably manipulated format. (WebKit is another beast, and an entire browser engine is far removed from a DOM, so any comparisons there are sketchy at best.)</p> <p>We don't and can't expect to hold non-UA component developers to the same standards as UA component developers, so it's hard to justify giving them the same carte blanche.</p> <blockquote> <p>The problem is that it quickly becomes library & framework author's problem once enough websites decide to take that risk.</p> </blockquote> <p>Beyond styling, it doesn't seem likely that many websites will depend on specific implementation details of specific components -- it's quite a headache to retrofit extras into an existing custom element so they're more likely to just modify it directly. For e.g. analytics libraries, or any of my other examples, the code is component-agnostic, and so the implementation can change without breaking anything.</p> <blockquote> <p>Don't use a badly written component, or a component you don't like. Nobody is forcing you to use a component that you don't like so much that you want to go in & modify its shadow tree.</p> </blockquote> <p>This argument looks reasonable at first, but once you consider vendor lock-in, things start to change. If you want to add e.g. twitter integration to your document and their component is under-specified, your only option is put up and shut up.</p> <blockquote> <p>page author can just copy / fork the component and modify its script not to use a shadow tree</p> </blockquote> <p>This seems like an inconvenient and roundabout way of achieving the same behaviour as my proposal, except without the fine-grained control. If the developer wants to make changes to the component, then yes, they should fork and modify. If they plan to use its DOM for any other purpose, having to modify the component for mere access seems unduly heavy-handed.</p> </div> </div> <div class="comment"> <div class="user"> <a rel="noreferrer nofollow" target="_blank" href="https://github.com/dylanb"><img src="https://avatars.githubusercontent.com/u/79245?v=4" />dylanb</a> commented <strong> 7 years ago</strong> </div> <div class="markdown-body"> <p>@matthewp </p> <blockquote> <p>I disagree completely @mrmr1993, closed shadow DOMs just give developers the same abilities that built-in elements have. If a new built-in form element were created then Vimium wouldn't work with it either. The solution here is to define an interface to interact with form elements that is generic enough that Vimium and others don't need to special case for every element type. These are the events to listen to changes and the properties to get/set the element's underlying value. There's no reason why if my custom element (with a closed shadow) emitted those events and had the correct properties why Vimium wouldn't just work with those.</p> </blockquote> <p>There are a bunch of conditions that hold for the standard HTML elements that do not hold for custom elements</p> <ol> <li> <p>Their APIs are very rich and standardized. The ways that other parts of the standard can interact with them is well thought through and relatively complete. They provide open, consistent and standardized APIs that allow interaction with them</p> </li> <li> <p>They can (for the most part) be relied upon to support accessibility APIs and keyboard interaction</p> </li> <li> <p>You can implement polyfills and other interactions purely in JavaScript without any limitations</p> </li> </ol> <p>All of these things mean that there are very few (I do not exclude the possibility, but cannot think of any) situations that have not already been identified (and are being worked on) where a document author would legitimately like to look inside and reach inside the shadow DOM.</p> <p>We cannot predict the ways the shadow DOM spec will be used by developers (just like we could not predict how HTML would be used by developers) and therefore, we cannot claim that the same will even remotely be possible with custom components that use shadow DOM.</p> </div> </div> <div class="comment"> <div class="user"> <a rel="noreferrer nofollow" target="_blank" href="https://github.com/rniwa"><img src="https://avatars.githubusercontent.com/u/285965?v=4" />rniwa</a> commented <strong> 7 years ago</strong> </div> <div class="markdown-body"> <blockquote> <p>I think the comparison with AppKit/UIKit and the WebKits is misleading. Since the AppKit/UIKit components are the lowest-level UI components (and the lowest-level, well-specified, reliable API to these components), the comparison should be with UA's native elements.</p> </blockquote> <p>We can agree to disagree on this point, and say this is a difference in our opinions.</p> <blockquote> <p>I'll happily agree with <a href="https://github.com/w3c/webcomponents/issues/640#issuecomment-301070698">@dylanb's statement</a> that UA elements <em>should not</em> expose their shadow DOMs, and by analogy that AppKit/UIKit should not expose their internals, since they are not described in a well-specified, reliably manipulated format. (WebKit is another beast, and an entire browser engine is far removed from a DOM, so any comparisons there are sketchy at best.)</p> <p>We don't and can't expect to hold non-UA component developers to the same standards as UA component developers, so it's hard to justify giving them the same carte blanche.</p> </blockquote> <p>Why not?</p> <blockquote> <blockquote> <p>Don't use a badly written component, or a component you don't like. Nobody is forcing you to use a component that you don't like so much that you want to go in & modify its shadow tree.</p> </blockquote> <p>This argument looks reasonable at first, but once you consider vendor lock-in, things start to change. If you want to add e.g. twitter integration to your document and their component is under-specified, your only option is put up and shut up.</p> </blockquote> <p>We already live in that world. If some social network only provides their social engagement widget in HTTP, for example, there is nothing you can do to make them use HTTPS instead. Also, social networks have incentives to make their widgets not customizable for branding purposes. Allowing random websites to allow changing the color of the buttons, etc... would be detrimental to their branding & marketing effort for example. Again, this should really be left to each social network site to decide. If one doesn't like the default UI provided by a social network, once could either not put their widget or use a hyperlink with a custom UI, etc...</p> <blockquote> <blockquote> <p>page author can just copy / fork the component and modify its script not to use a shadow tree</p> </blockquote> <p>This seems like an inconvenient and roundabout way of achieving the same behaviour as my proposal, except without the fine-grained control. If the developer wants to make changes to the component, then yes, they should fork and modify. If they plan to use its DOM for any other purpose, having to modify the component for mere access seems unduly heavy-handed.</p> </blockquote> <p>This whole argument hinges on the existence of such a convincing use case in which developers try to access and mutate shadow tree, yet nobody has given one so the premise of the statement has not been well established.</p> </div> </div> <div class="comment"> <div class="user"> <a rel="noreferrer nofollow" target="_blank" href="https://github.com/dylanb"><img src="https://avatars.githubusercontent.com/u/79245?v=4" />dylanb</a> commented <strong> 7 years ago</strong> </div> <div class="markdown-body"> <p>Here is a very useful polyfill that will not work for custom components with closed shadow DOM <a href="https://github.com/WICG/inert/blob/master/src/inert.js">https://github.com/WICG/inert/blob/master/src/inert.js</a></p> </div> </div> <div class="page-bar-simple"> <a href="/WICG/webcomponents/640?page=2" class="next">Next</a> </div> <div class="footer"> <ul class="body"> <li>© <script> document.write(new Date().getFullYear()) </script> Githubissues.</li> <li>Githubissues is a development platform for aggregating issues.</li> </ul> </div> <script src="https://cdn.jsdelivr.net/npm/jquery@3.5.1/dist/jquery.min.js"></script> <script src="/githubissues/assets/js.js"></script> <script src="/githubissues/assets/markdown.js"></script> <script src="https://cdn.jsdelivr.net/gh/highlightjs/cdn-release@11.4.0/build/highlight.min.js"></script> <script src="https://cdn.jsdelivr.net/gh/highlightjs/cdn-release@11.4.0/build/languages/go.min.js"></script> <script> hljs.highlightAll(); </script> </body> </html>