anywidget getSelection browser API not working

[metaboulie]

Describe the bug

Colab link of this anywidget

Here is a comparison of an anywidget I built in Jupyter and marimo.

Jupyter

marimo

The main reason may be that the html styles are not read and rendered correctly in marimo.

Environment

{
  "marimo": "0.8.7",
  "OS": "Darwin",
  "OS Version": "22.3.0",
  "Processor": "arm",
  "Python Version": "3.12.5",
  "Binaries": {
    "Browser": "--",
    "Node": "v22.7.0"
  },
  "Requirements": {
    "click": "8.1.7",
    "importlib-resources": "missing",
    "jedi": "0.19.1",
    "markdown": "3.7",
    "pymdown-extensions": "10.9",
    "pygments": "2.18.0",
    "tomlkit": "0.13.2",
    "uvicorn": "0.30.6",
    "starlette": "0.38.4",
    "websockets": "12.0",
    "typing-extensions": "4.12.2",
    "ruff": "0.6.3"
  }
}

Code to reproduce

import marimo

__generated_with = "0.8.7"
app = marimo.App(width="medium")

@app.cell(hide_code=True)
def __():
    import marimo as mo
    return mo,

@app.cell(hide_code=True)
def __():
    import anywidget
    import traitlets
    import json

    class MultiTextAnnotationWidget(anywidget.AnyWidget):
        _esm = """
        function render({ model, el }) {
            const data = model.get("data");
            const labels = model.get("labels");
            let currentTextIndex = 0;

            // Create a container for the text
            let textContainer = document.createElement("div");
            textContainer.className = "text-container";
            el.appendChild(textContainer);

            // Create a container for the legend
            let legendContainer = document.createElement("div");
            legendContainer.className = "legend-container";
            el.appendChild(legendContainer);

            // Create navigation buttons
            let navContainer = document.createElement("div");
            navContainer.className = "button-container";
            let prevButton = document.createElement("button");
            prevButton.innerText = "previous";
            prevButton.className = "nav-button";
            prevButton.addEventListener("click", () => navigateText(-1));
            let nextButton = document.createElement("button");
            nextButton.innerText = "next";
            nextButton.className = "nav-button";
            nextButton.addEventListener("click", () => navigateText(1));
            navContainer.appendChild(prevButton);
            navContainer.appendChild(nextButton);
            el.appendChild(navContainer);

            // Create a container for the labels
            let labelContainer = document.createElement("div");
            labelContainer.className = "button-container";
            for (let [label, color] of Object.entries(labels)) {
                let labelButton = document.createElement("button");
                labelButton.innerText = label;
                labelButton.style.backgroundColor = color;
                labelButton.className = "label-button";
                labelButton.addEventListener("click", () => addAnnotation(label, color));
                labelContainer.appendChild(labelButton);
            }
            el.appendChild(labelContainer);

            // Create a remove button (initially hidden)
            let removeButton = document.createElement("button");
            removeButton.innerText = "Remove";
            removeButton.style.display = "none";
            removeButton.className = "remove-button";
            removeButton.addEventListener("click", removeAnnotation);
            el.appendChild(removeButton);

            let currentAnnotation = null;

            function navigateText(direction) {
                currentTextIndex += direction;
                if (currentTextIndex < 0) currentTextIndex = data.length - 1;
                if (currentTextIndex >= data.length) currentTextIndex = 0;
                updateTextDisplay();
            }

            function updateTextDisplay() {
                let currentResult = JSON.parse(model.get("result"));
                let currentAnnotations = currentResult[currentTextIndex] || [];
                let text = data[currentTextIndex];

                // Sort annotations by start position in descending order
                currentAnnotations.sort((a, b) => b.start - a.start);

                // Apply annotations
                for (let annotation of currentAnnotations) {
                    let before = text.slice(0, annotation.start);
                    let annotated = text.slice(annotation.start, annotation.end);
                    let after = text.slice(annotation.end);
                    text = before +
                           `<span class="annotation" style="background-color: ${labels[annotation.label]};" data-id="${annotation.id}" data-label="${annotation.label}">` +
                           annotated +
                           '</span>' +
                           after;
                }

                textContainer.innerHTML = text;
                updateLegend(currentAnnotations);
            }

            function updateLegend(annotations) {
                let legendHTML = '';
                for (let [label, color] of Object.entries(labels)) {
                    let annotationsForLabel = annotations.filter(a => a.label === label);
                    let annotatedTexts = annotationsForLabel.map(a => a.text).join(', ');
                    legendHTML += `
                        <div class="legend-row">
                            <div class="legend-label">
                                <span class="legend-color" style="background-color: ${color};"></span>
                                ${label}
                            </div>
                            <div class="legend-text">${annotatedTexts}</div>
                        </div>
                    `;
                }
                legendContainer.innerHTML = legendHTML;
            }

            function addAnnotation(label, color) {
                let selection = window.getSelection();
                if (selection.rangeCount > 0) {
                    let range = selection.getRangeAt(0);
                    let selectedText = range.toString();
                    if (selectedText) {
                        let start = getTextPosition(range.startContainer, range.startOffset);
                        let end = getTextPosition(range.endContainer, range.endOffset);

                        let annotationId = `${currentTextIndex}-${Date.now()}`;

                        let currentResult = JSON.parse(model.get("result"));
                        if (!currentResult[currentTextIndex]) {
                            currentResult[currentTextIndex] = [];
                        }
                        currentResult[currentTextIndex].push({
                            id: annotationId,
                            text: selectedText,
                            label: label,
                            start: start,
                            end: end
                        });
                        model.set("result", JSON.stringify(currentResult));
                        model.save_changes();

                        updateTextDisplay();
                    }
                }
            }

            function getTextPosition(node, offset) {
                let position = 0;
                let walker = document.createTreeWalker(textContainer, NodeFilter.SHOW_TEXT, null, false);

                while (walker.nextNode()) {
                    if (walker.currentNode === node) {
                        return position + offset;
                    }
                    position += walker.currentNode.length;
                }

                return position;
            }

            function removeAnnotation() {
                if (currentAnnotation) {
                    let currentResult = JSON.parse(model.get("result"));
                    let textAnnotations = currentResult[currentTextIndex];
                    let index = textAnnotations.findIndex(item => item.id === currentAnnotation.id);
                    if (index > -1) {
                        textAnnotations.splice(index, 1);
                        model.set("result", JSON.stringify(currentResult));
                        model.save_changes();
                    }

                    currentAnnotation = null;
                    removeButton.style.display = "none";

                    updateTextDisplay();
                }
            }

            textContainer.addEventListener("mouseup", (event) => {
                let target = event.target;
                if (target.classList.contains("annotation")) {
                    currentAnnotation = {
                        id: target.dataset.id,
                        label: target.dataset.label
                    };
                    removeButton.style.display = "inline-block";
                } else {
                    currentAnnotation = null;
                    removeButton.style.display = "none";
                }
            });

            updateTextDisplay();
        }
        export default { render };
        """
        _css = """
        .text-container {
                    border: 1px solid #444;
                    padding: 15px;
                    margin-bottom: 10px;
                    font-family: Arial, sans-serif;
                    font-size: 16px;
                    line-height: 1.5;
                    background-color: #2a2a2a;
                    color: #f0f0f0;
                    border-radius: 5px;
                    white-space: pre-wrap;
                }
                .legend-container {
                    margin-top: 10px;
                    padding: 10px;
                    background-color: #333;
                    border-radius: 5px;
                }
                .legend-row {
                    margin-bottom: 5px;
                    display: flex;
                    align-items: flex-start;
                }
                .legend-label {
                    display: flex;
                    align-items: center;
                    width: 120px;
                    margin-right: 10px;
                }
                .legend-color {
                    display: inline-block;
                    width: 12px;
                    height: 12px;
                    margin-right: 5px;
                    border-radius: 2px;
                }
                .legend-text {
                    flex-grow: 1;
                    font-size: 14px;
                }
                .nav-button, .label-button, .remove-button {
                    background-color: #007bff;
                    color: white;
                    border: none;
                    padding: 10px 15px;
                    margin: 5px;
                    cursor: pointer;
                    border-radius: 5px;
                    font-size: 14px;
                    transition: background-color 0.3s;
                }
                .nav-button:hover, .label-button:hover, .remove-button:hover {
                    background-color: #0056b3;
                }
                .annotation {
                    cursor: pointer;
                    transition: opacity 0.3s;
                }
                .annotation:hover {
                    opacity: 0.7;
                }
                .button-container {
                    margin-bottom: 10px;
                }
        """
        data = traitlets.List().tag(sync=True)
        labels = traitlets.Dict().tag(sync=True)
        result = traitlets.Unicode().tag(sync=True)

        def __init__(self, data, labels):
            super().__init__()
            self.data = data[:]
            self.labels = labels
            self.result = json.dumps({i: [] for i in range(len(data))})
    return MultiTextAnnotationWidget, anywidget, json, traitlets

@app.cell(hide_code=True)
def __(MultiTextAnnotationWidget, mo):
    # Example usage
    data = [
        "This is the first text that you can annotate.",
        "Here's a second text for annotation.",
        "And a third one to demonstrate multiple texts.",
    ]
    labels = {"Important": "red", "Question": "blue", "Note": "green"}

    widget = mo.ui.anywidget(MultiTextAnnotationWidget(data=data, labels=labels))
    return data, labels, widget

@app.cell
def __(widget):
    widget
    return

@app.cell
def __(widget):
    widget.result
    return

if __name__ == "__main__":
    app.run()

[mscolnick]

This is because you are adding the styles outside of the widget: document.head.appendChild(style);.

We use shadow-doms with widgets so they don't pollute the global css space, which Jupyter does not. There is a _css field on the widget that you should use for styles.

[mscolnick]

Feel free to re-open if there are other issues

[metaboulie]

I've moved the css to _css, however, the annotation action -- select a range of words, click one of the buttons below doesn't work

[mscolnick]

Looks like an issue with getSelection. I don't think ShadowRoots support selection consistently today in browsers. Each browser seems to implement it their own way:

https://stackoverflow.com/questions/62054839/shadowroot-getselection

The state of affairs as of Dec 2023:

ShadowRoot.getSelection is a non-standard API.

Selection.getComposedRanges is a standards proposal to support selection with Shadow DOM.

On Chromium, calling document.getSelection will not pierce into the Shadow DOM and gives you some unhelpful high-level element. But it does expose the non-standard getSelection method on the ShadowRoot.

On Firefox, it does not implement ShadowRoot.getSelection, but document.getSelection will pierce through shadow dom and give you the exact element.

On Safari, Selection.getComposedRanges is supported as of v17. On versions before that, ShadowRoot.getSelection is not supported and apparently document.getSelection does not pierce the Shadow DOM, meaning you are just out of luck.

[mscolnick]

@metaboulie i can leave this open to track for now. others may be able to post workarounds