Make it possible to observe connected-ness of a node

[rniwa]

It would be useful to have a way of observing when a node gets connected or disconnected from a document similar to the way connectedCallback and disconnectedCallback work for custom elements but unlike those at the end of the microtask like all other mutation records.

[rniwa]

@smaug---- @justinfagnani

[domenic]

It wasn't clear from the original thread what the use cases were. @pemrouze just said it would be nice, then people started discussing how. We never even talked about why they couldn't use custom elements.

I agree that if we had a need for this ability extending MutationObserver would be the way forward but I'd like some more justification in the form of concrete use cases that are hard to accomplish with custom elements.

[rniwa]

Agreed. It's always good to have a list of concrete use cases before adding new API.

[smaug----]

IIRC this was discussed around the time when MutationObserver was designed and the idea was that since one can add childList observer to the document and then just check document.contains(node), that should in principle be enough (and the things to observe was kept in minimum). However Shadow DOM wasn't really a thing at that time, and that may have changed the needs. Concrete use cases, which can't be solved with Custom Elements, would be indeed nice. The more we add APIs, the more complicated platform we have to maintain and adding more and more API tends to slow down implementations, or at least makes it harder to optimize them.

[justinfagnani]

This would be useful for polyfilling custom elements, but obviously that's not a good use case for why custom elements aren't enough.

If I can page in my requests around MutationObservers one had to do with observing access shadow boundaries so we don't have to patch attachShadow, which might be relevant here, and have some way of unobserving nodes outside of disconnect(), say when the nodes are disconnected.

[pemrouz]

It wasn't clear from the original thread what the use cases were. @pemrouze just said it would be nice, then people started discussing how. We never even talked about why they couldn't use custom elements.

• A framework that does some housekeeping when a component is removed (e.g. tearing down subscriptions). The framework is outside the component, so would have to rely on every component emitting this event manually which isn't feasible/reliable. The disconnectedCallback is the current solution, but it requires wrapping every component in a wrapper component. This has other benefits (e.g. dynamic registry), but the less that can be done at the framework level would be great and this seemed generic enough to enquire about.

• It would be nice for a component to be able to declaratively manage subscriptions it's own subscriptions, rather than have to emit the event manually or be forced to imperatively manage the subscription (see original thread).

[smaug----]

Wouldn't a childList observer in document take care of those use cases, at least in common cases?

[pemrouz]

It's unfortunately notoriously difficult to do with MutationObserver, and pretty much guaranteed to be buggy (due to it being batched) and inefficient (having to watch every change on the entire document).

[WebReflection]

This would be useful for polyfilling custom elements

polyfills for custom elements already work and are either based on MutationObserver or DOMNodeInserted/DOMNodeRemoved with a lookup for registered elements in the list of changed nodes.

Using polyfills is a weak reason, specially because anythig new needs to be polyfilled too.

However I do have a use case in hyperHTML too but I believe React and many other frameworks would love this feature as well.

CustomElements mechanism to understand their life-cycle are excellent but not everyone writes Custom Elements based sites and knowing a generic form, input, list element, grid card, has been added/removed would be ace.

in hyperHTML I do have control of each vanilla DOM component but I need to do this "dance" per each MutationObserver notification:

function dispatchTarget(node, isConnected, type, e) {
    if (components.has(node)) {
      node.dispatchEvent(e || (e = new Event(type)));
    }
    // this is where it gets a bit ugly
    else {
      for (var
        nodes = node.children,
        i = 0, length = nodes.length;
        i < length; i++
      ) {
        e = dispatchTarget(nodes[i], isConnected, type, e);
      }
    }
    return e;
  }

The dispatching after recursive search is done per each record.addedNodes and record.removedNodes but if I had a platform way to loop over all known connected/disconnected nodes my code, and my life, would definitively be easier.

[pemrouz]

@domenic any more thoughts?

[rniwa]

Here's a concrete use case. A editor library wants to know when the editor lost the focus. Because blur event does not fire on an element that got disconnected from a document before losing the focus, we'd need to monitor when the element is disconnected. To do this, we currently need to monitor all child node changes of the parent node, and then detect when the editor node is removed. With the proposed observation type, we can observe the disconnection of the editor element instead.

[WebReflection]

@rniwa what you described looks more like a blur related bug (either browser or specs) but I'm happy you found a concrete use case.

Another one I have these days is a declarative way for an element inside a template to perform setup operations without needing its container to know about these

<tempalte>
  <div>
    One <span>or more</span> elements and then
   <p
    class="timer"
    onconnected="this.textContent = (new Date).toLocaleString()"
  > time </p>
  </div>
</template>

[smaug----]

That example doesn't use MutationObserver at all. onconnected sounds like a Mutation Event.

[rniwa]

blur event is not supposed to fire when an element is removed. Due to security concerns, we can't fire blur synchronously in that case.

[WebReflection]

@smaug you're right, that example is not directly related but it's something I can do already and it's based on MutationObserver. The onconnected is just a hint, imagining a list of added nodes per each record could be checked against node.hasAttribute("onconnected") and react accordingly.

That being said I wouldn't mind having a native onconnected/disconnected event in the DOM though but that's another story.

---

@rniwa

Due to security concerns, we can't fire blur synchronously in that case.

I am having hard time understanding why there wouldn't be security concern if blur can be intercepted through an observer ... what's the difference?

A focused node that never fires blur is a source of troubles. Mitigating troubles through other mechanism is a work around, not a solution to that issue.

Regardless, if that's the case you were looking for to justify connected/disconnected, I'm OK with it.

[rniwa]

MutationObserver fires at the end of the current micro-task, that's much later than synchronous timing which poses a security threat in our engine.

[wessberg]

Yes, this is much needed indeed. Tracking when elements are connected/disconnected is rather easy with MutationObservers when not using Shadow DOM, but with it and as I see it, we have to observe the childList of the root of all parents going recursively up the tree from the node in question. The parent chain of the node may change or the node may have no parent(s) by the time the MutationObservers are configured, so this won't even fully solve the problem. Looking back at MutationEvents, from my own testing, DOMNodeRemovedFromDocument doesn't seem to fire if a child element is indirectly removed from the DOM if it is located within the ShadowRoot of another host element that is disconnected.

I have a concrete use case for progressive enhancement that is much inspired by the Custom Attributes proposal that was discussed a while back and I've brought into a user-land library. Custom Attributes are sort of like is, except they can annotate any element, receive values, and implement the Custom Elements lifecycle hooks for it. Think of it as decorators for HTML, or behaviors.

For example:

// Listens for PointerEvents and paints Material Design Ripples on the host element
class Ripple {
    constructor (hostElement) {...}
    connectedCallback () {...}
    disconnectedCallback () {...}
    // ...
}
customAttributes.define("*ripple", Ripple);

<!-- Custom Attributes are element-agnostic -->
<button *ripple>Click me</button>
<!-- They can also receive semi-colon separated key-value pairs as values, just like the style attribute -->
<my-element *ripple="center: true"></my-element>

Now, while I would like to see Custom Attributes/mixins/behaviors standardized since is for Customized Built-In Elements isn't agnostic of host elements, that is a discussion for another place, but nevertheless it shows a concrete use case.

To solve this today, using MutationObservers just doesn't cut it when using Shadow DOM. There may be many, changing, roots going up the tree from the node, or there may be none at all! You can observe child lists of all roots and hosts from the node and up the tree, but you have to do a lot of work to track changes to the parent chain and update the observers. But if the node has no parent by the time the MutationObserver needs to be configured, you will never know which root to observe.

The "best" way to do this right now (as I see it) is monkey patching the Node and Element prototypes, similar to what the Polymer team does in the Custom Elements polyfill, and then check connection states of affected notes within those operations. For anything else than polyfills, I think this is bad for performance, and it is synchronous, something MutationObservers were designed in part to avoid, and basically just something to be avoided if possible. Which is why I'm strongly rooting for extending MutationObserverOptions with a connectionState property or something like it. That way, the asynchronous nature is preserved and monkey patching is avoided.

[caridy]

I would like to revamp this conversation. It seems that we have more use-cases that can help us to come to an agreement about this feature request. Maybe adding it to the agenda for the F2F meeting.

[rniwa]

Yeah, I think we should just add this. Here's a concrete proposal:

Add boolean connectedness to MutationObserverInit, which if set would make the mutation observer start receiving a mutation record whenever the target node is connected to or disconnected from a document.

Add two mutation record types connected and disconnected to MutationRecord. connected is used the target node is newly connected to a document. disconnected is used when the target node has been is newly removed from a document.

subtree option in mutation observer would the observation subtree-wide.

[wessberg]

Sounds great. Wouldn't it be sufficient with a simple boolean connected which is true|false?

[rniwa]

Sounds great. Wouldn't it be sufficient with a simple boolean connected which is true|false?

Sorry, I wasn't thinking through. We could just add new mutation record types like connected / disconnected and not even add a boolean. Updated the proposal accordingly.

[trusktr]

We never even talked about why they couldn't use custom elements.

@domenic The main use case I imagine is with builtins, so we can get connected/disconnected behavior that behaves the same as the callbacks for Custom Elements do. (I didn't see this thread before I wrote https://github.com/w3c/webcomponents/issues/785)

It'd be useful in some of the other issues I've had troubles with too.

It's always good to have a list of concrete use cases before adding new API.

@rniwa Another use case is simply: someone else made a web app, now we want to manipulate it without necessarily touching their source code (maybe we don't have access to the source, maybe we are writing a browser plugin, etc). There's still the issue of reaching into closed shadow trees though (it still requires patching attachShadow).

In a sense the purpose of this feature is more similar to the purpose of jQuery: manipulate existing DOM elements with it. This is in contrast to the act of designing the behaviors of our own DOM elements (custom elements), instead we can use this to manipulate existing DOM elements (builtin or custom).

👍 to this.

[caridy]

@rniwa here is our update on this:

today, there is no way to observe the synchronous connection and disconnection of a particular element, the only option is MutationObserver (which is async), or registering a custom element (which sometimes it is not an option or overkill).

Just to clarify, in the meeting @domenic mentioned Mutation Events, (e.g.: DOMNodeInserted works in all browsers today but does NOT work inside a shadow, and doesn't have the same semantics as connectedCallback). Additionally, DOMNodeInsertedIntoDoc does have the same semantics, but does not work in all browsers, neither inside a shadow).

The only real solution today is a very hacky solution, same solution used by us and by the polymer custom element polyfill, which is to patch all DOM APIs that allow you to mutation the childNodes, so you can insertion and removal of elements, to simulate the synchronous connect/disconnect callback semantics.

My question is, can we get a reliable way to listen for this node reaction, synchronous, without registering the element?

[wessberg]

@caridy, I'm not sure if I believe a synchronous approach will be able to move forward since we had that with Mutation Events (which as you say doesn't work reliably with Shadow roots) and one of the primary reasons to move away from them in favor of MutationObserver was their synchronous nature. I'm certain you have perfectly valid reasons to want a synchronous solution, and I agree that this is an important missing piece in the web platform right now, but if we can find a way to move forward with an asynchronous approach and with an API that is similar to other related observers such as MutationObserver, I think the odds of getting implementors behind this is greater. I've experimented with this and created a library called ConnectionObserver which batches connection entries together in microtasks, and I'd like to propose something similar to that.

[annevk]

@wessberg how is https://github.com/whatwg/dom/issues/533#issuecomment-405530738 solved by extensions to MutationObserver? You'd still not know about shadow tree mutations.

Having reread the thread it's not entirely clear to me everyone is on the same page here when it comes to use cases and requirements, which might also be why we keep hearing different solutions being proposed.

Starting from the top at https://whatwg.org/faq#adding-new-features would be beneficial I think.

[wessberg]

@annevk, if MutationObserver is extended with a connectedness option, it would no longer be relevant to track Shadow tree mutations. Instead, each time the observed node is attached to or detached from the DOM, a MutationRecord is created holding a boolean value indicating whether or not the node is connected.

However, alternatively, we could consider putting this functionality into an entirely new observer as described in my previous comment. This observer would observe a node, and be agnostic of whatever root it lives within.

An example of using such API could be:

// Hook up a new ConnectionObserver
const observer = new ConnectionObserver(entries => {
    for (const {connected, target} of entries) {
        console.log("target:", target);
        console.log("connected:", connected);
    }
});

// Observe 'someElement' for connectedness
observer.observe(someElement);

// Eventually disconnect the observer when you are done observing elements for connectedness
observer.disconnect();

More specifically:

class ConnectionObserver {
    [Symbol.toStringTag]: string;

    /**
     * Constructs a new ConnectionObserver
     * @param {ConnectionCallback} callback
     */
    constructor(callback: ConnectionCallback);

    /**
     * Observes the given target Node for connections/disconnections.
     * @param {Node} target
     */
    observe(target: Node): void;

    /**
     * Returns the entries that are currently queued in the batch  and haven't been processed yet,
         * leaving the connection queue empty. This may be useful if you want to immediately fetch all
         * pending connection records immediately before disconnecting the observer, so that any
         * pending changes can be processed.
     * @return {ConnectionRecord[]}
     */
    takeRecords(): ConnectionRecord[];

    /**
     * Disconnects the ConnectionObserver such that none of its callbacks will be invoked any longer
     */
    disconnect(): void;
}

// A ConnectionCallback must be provided to the constructor of ConnectionObserver and will be invoked when there are new ConnectionRecords available.
type ConnectionCallback = (entries: ConnectionRecord[], observer: ConnectionObserver) => void;

// ConnectionCallbacks are invoked with an array of ConnectionRecords. Those have the following members:
interface ConnectionRecord {
    /**
     * Whether or not the node is Connected
     */
    readonly connected: boolean;

    /**
     * The target Node
     */
    readonly target: Node;
}

That said, I think putting connectedness as an option for MutationObserver is just as fine.

@annevk, I think you are right about starting fresh in regards to describing what problems we are trying to solve, and how they are related.

I would say it boils down to generalizing the functionality enabled by the connectedCallback and disconnectedCallback lifecycle hooks of Custom Elements and bringing this functionality to any DOM node (including Text nodes, Comments, etc) to make it possible to react to connectedness changes in nodes you don't necessarily control, and isn't necessarily Custom Elements.

I'm against solving it by moving the two lifecycle callbacks into the Node interface for several reasons, including that they are synchronous and expensive.

I'm in favor of an observer approach since multiple observers will be able to observe the same node, and most importantly, there are no performance impact for nodes for which there is no associated observer.

[caridy]

@annevk I'm not proposing any particular solution, just trying to describe why we need this, and what are the options today that do not fulfill those use-cases.

To be more specific about use-cases, I have one main user-story for the sync version of the connect-ness detection:

as a developer, I can emulate what you can do with a custom element callbacks without requiring to use a custom element.

The use-cases are various, here are a few:

• frameworks and libraries that do not use custom elements, requires a recursive search on subtrees to trigger their own disconnect hook even though not all elements use that hook.
• frameworks and libraries that create node-trees during the diffing algo, and later on install them at once into the document, will have to either a) trigger the connect hook when the element is added to its parent (even though it is not really in the doc yet - usually this is the solution), or b) recursively search in the virtual tree for vnodes that requires connect hook, to trigger that after the root was inserted into the dom, or c) never do node-trees in memory, always create them in the actual document (which has perf implications).
• polyfilling custom elements, or emulating custom elements in general, is impossible without hacking many APIs today.

[annevk]

So by synchronous you mean equivalent timing to custom elements, not equivalent timing to mutation events? Why is mutation observer timing insufficient?

Why would you need to polyfill/emulate custom elements at this point?

[caridy]

@annevk

So by synchronous you mean equivalent timing to custom elements, not equivalent timing to mutation events? Why is mutation observer timing insufficient?

For the same reasons node reactions are sync for CE callbacks. you want to know when the element is connected/disconnected to take immediate action, from registration, to coordinations with parent, to any other thing that all these libraries and frameworks are doing today synchronous as part of the diffing algo. Good example here is the snabbdom library that powers VUE and LWC, it is one of those popular diffing libs that provides tons of sync hooks for the operations on the dom.

Why would you need to polyfill/emulate custom elements at this point?

Polyfilling is always handy, we don't know what the future will bring :), one example here is the scoped registry, that will take a while to take shape, but having this feature in place means that we can emulate some of the custom elements reactions without registering a real CE. For the record, we do that today in LWC, but it requires many hacks to control the connect-ness, so I might be bias. But in general, having a way to emulate these low level reactions seems like a good feature to have.

[wessberg]

@caridy, if we are going to pursue something that guarantees CE reaction timing for any given element, then I suggest opening a separate thread with such a proposal, because this thread so far has been centered around primarily MutationObserver which is asynchronous in it's timing (and isn't restricted to just Elements, but any descendant of the Node interface). All of the Observer APIs are asynchronous, so I would argue that a sync solution would require a different approach such as callbacks or events, but in regards to events, we've already been down that road. If every node is to emit events when their connectedness changes, I would assume that would be very expensive. If we go with callbacks, then we lose multicast functionality (more than one observer may be interested in observing the connectedness of any given node).

I'm getting the feeling that this proposal is on the path to stagnation if we don't either:

• (A) split it up into two separate proposals
• (B) resolve the sync/async debate and proceed from there.

[caridy]

@wessberg I'm fine with creating a different issue. The only reason why we keep debating this here is because in the last F2F meeting we discussed this topic (from the agenda), and the point of contention was exactly what you mentioned, sync vs async.

As for the perf penalties, I will like to get more clarification from implementers. In theory, and by looking at the spec, the node reaction mechanism is in place for all elements, we only tap into them for custom elements today. The old mutation events are not necessarily a good reference, and they semantics are confusing, but certainly something to keep in mind during this discussions. @rniwa do you want me to open another issue for the sync proposal?

[rniwa]

Let's not open a new issue. We should figure out the best way to observe the connected-ness of a node in this issue.

[wessberg]

@rniwa alright. To avoid stalling, how about we investigate whether synchronous reactions are feasible first?

@caridy, you say that you would like get feedback from implementors in regards to the perf implications of sync reactions. I think that is a great idea since it helps us understand if a sync solution is feasible. If implementors will simply block this proposal out of performance concerns and lessons learned from Mutation Events, that would be a shame. Is there anyone reading this who can chime in here or page someone who can?

I think a way forward for us is to first understand what we can all agree on in regards to goals/non-goals.

Is everone in agreement that:

• We're concerned with the connectedness of any node, and not just elements?
• We want multiple observers to be able to observe the connectedness of a node simultaneously?

[caridy]

hey implementers, any feedback on the implications of sync reactions?

@wessberg as for your two bullet points, my assumption is that yes, we want to also observe nodes, and yes, we want multiple actors to observe the connectedness of the node at once.

[Jack-Works]

If I understand this correctly, connected-ness means if a Node appears in the document or not. If so, I'd like to share my use case. I have a span element and I want to mount my application on it when the span has been inserted in the document (I cannot control when it will be mounted). If I mount the application on the span when it is disconnected, it will throw an error cause most of our code doesn't assume that.

Now I have two choices, attach an event listener of deprecated DOMNodeInserted on the span, or add a childList MutationObserver on the whole document then use document.contains. I don't know which one hurts the performance mostly and they both don't seem like a good option.

[annevk]

Why can you not use a custom element?

[Jack-Works]

Why can you not use a custom element?

Our application is a Web Extension, which customElements registry is no available for us.

Web extension cannot register custom elements in content scripts.

[WebReflection]

@Jack-Works I think this polyfill would solve your issue. Alternatively, element-notifier seems to address this exact issue too.

[wessberg]

@Jack-Works I think this polyfill would solve your issue. Alternatively, element-notifier seems to address this exact issue too.

This section in the README of your element-notifier library is the most important reason why, as I've argued in this thread, MutationObserver in its current state is not currently sufficient in terms of addressing this, as the tree may have multiple roots.

Your library works very similarly to the ConnectionObserver library that I built to address this, and it works across shadow roots, but it comes at the cost of patching Element.prototype.attachShadow which is something I really hope can be avoided in the future.

It is unfortunate that this discussion hit a dead-end because we couldn't come to agreement about the whole sync vs async thing. I am still very much of the opinion that this thing has to be asynchronous and follow the same patterns as other existing observers, as a sync approach will most likely trigger performance concerns among implementers, just like MutationEvents did.

[jfbrennan]

Just want to add my use case:

<template>
  <p>My Custom Element with a menu</p>
  <button onclick="toggleMenu">Toggle</button>
  <x-menu>...</x-menu>
</template>

That template gets used in a Custom Element where I also call a utility function that takes the custom element as its one param, searches all its children looking for any onevent attributes which came from the template, and replaces them by registering an event listener - element.addEventListener('click', e => this.toggleMenu(e)) - and removes the event property. In other words, template event binding.

I'd like this utility function to also remove the listeners when the custom element is disconnected. If the utility class had a way to know when the custom element was disconnected that would be possible. Right now I have to return the collection of event registrations to the custom element and it runs through that list in its disconnectedCallback.

[LeaVerou]

Not sure if it has been suggested, but one way to make this plus a few more use cases easier would be to add an option to monitor parent changes. It's simpler than the original problem, and in some ways more general. It would still be annoying to monitor connected-ness, but a lot less so (a utility function instead of a whole library).