Provide better developer experience for ReactiveHTML

MarcSkovMadsen commented 3 years ago

I believe ReactiveHTML is important a it makes interacting with javascript from python easy. It opens up for more pythonistas being able to use and contribute. The power is its simplicity and "hot reloading". And the components work really well and snappy.

BUT 😄

As soon as you spend more than 5-10 minutes on a ReactiveHTML model you start experiencing the downsides.

A .py file is not really meant for developing javascript. You don't get any help from your editor or IDE.
It's hard to develop something that is not tightly coupled to Panel and the ReactiveHTML model and could easily be carved out and used in a broader context.

I would like to be able to develop more general, reusable js functions, Web Components or React Components that we can use in 20 years when we develop Rusty - the most powerful data app framework for Rust 😄 For example if I invest the time in building the Fast components for Panel I would like to develop them separately and then bind them to Panel as simple as possible.

I would like to be able to keep the _scripts writing to a minimum. One liners hooking up to the external js is ok. But if they could even be avoided it would be awesome.

I would like to be able to easily distribute this as a part of my package without using advanced js build tools and distribute with npm/ unpackage etc.

Maybe the solutions are already there? But have to be documented or explained?

Maybe avoiding the build tools is a bad idea anyways for somebody who wants to build something larger and reusable. I just like the simplicity right now and the autoreload with Panel.

Potential Solutions

Local Javascript File

If we could refer to a local .js file in __javascript__ or __js_modules__ or in a new __raw_javascript__/ __local_javascript__ that might solve the problem. Especially if it worked with --autoreload. If we even could implement self there for the ReactiveHTML model I think it would be awesome.

Bokeh model infrastructure

Maybe the right way to go for building a larger project or distributing a package is hooking in to the existing Bokeh infrastructure for building models. It would be nice to see an example though where the infrastructure can be used but without building bokeh models.

MarcSkovMadsen commented 3 years ago

My colleague showed me what he did

To me just shows that I am not the only one thinking the developer experience could be improved. But I think reading in a separate file for each key, value in _scripts is overkill. It will lead to mental overhead and slow down the project.

MarcSkovMadsen commented 3 years ago

Some way of using .js files would be nice.

Something for the external shared utility functions/ general components. And something for the specific scripts.

import panel as pn
import param

class MyClass(pn.reactive.ReactiveHTML):
    value = param.String()

    __javascript__ = ["general_scripts.js"]
    _scripts = "scripts.js"

general_scripts.js

my_func = function(value){
  console.log(value)
}

scripts.js

render = () => {
  console.log(data)
}
value = () => {
  my_func(value)
}

Remember:

Having this work with --autoreload is key for a fast developer experience.
Having the ability to put the general script in __javascript__ or __javascript_modules__ is key.

MarcSkovMadsen commented 3 years ago

Something like the below read_scripts function can read a .js with a specific format and convert it to the _scripts dict.

"""Module for making the developer experience for ReactiveHTML
event better
"""
import pathlib
import textwrap
from typing import Dict

def _clean_script(value):
    return textwrap.dedent(value).strip()

def text_to_scripts(text: str) -> Dict:
    """Returns a `_scripts` dictionary for ReactiveHTML based on a string

    Args:
        text (str): The input string

    Returns:
        Dict: The output `_scripts`

    Example:

    >>> txt='''
    ... render=()=>{
    ...   console.log(data)
    ... }
    ... value=()=>{
    ...   my_func(value)
    ... }
    ... '''
    >>> text_to_scripts(txt)
    {'render': 'console.log(data)', 'value': 'my_func(value)'}
    """
    lines = text.split("\n")
    scripts = {}
    key = ""
    value = ""
    for line in lines:
        if key:
            if line == "}":
                scripts[key] = _clean_script(value)
                key = value = ""
            else:
                value += line + "\n"
        else:
            if line and not line[0] == " " and line.endswith(")=>{") and "=(" in line:
                key = line.split("=")[0]

    return scripts

def read_scripts(jsfile: str = "", pyfile: str = "") -> dict:
    """Reads a `.js` file and converts it to a _scripts dictionary

    Args:
        jsfile (Union[str,pathlib.Path], optional): The path or name of the file.
        pyfile (str, optional): Optional __file__ of the .py file file.
            If provided its assumed the jsfile is in the same folder as pyfile

    Returns:
        dict: A dictionary of _scripts for a ReactiveHTML _template
    """
    if pyfile:
        full_path = pathlib.Path(pyfile).parent / jsfile
    else:
        full_path = pathlib.Path(jsfile)

    with open(full_path, "r", encoding="utf8") as _file:
        text = _file.read()
    return text_to_scripts(text)

import pytest

EXAMPLE_SCRIPTS = [
    (  # Reference Example
        """
render=()=>{
  console.log(data)
}
value=()=>{
  my_func(value)
}
""",
        {"render": "console.log(data)", "value": "my_func(value)"},
    ),
    # With and without function arguments
    (
        """
theme=()=>{
    state.chart.updateOptions({theme: {mode: state.tMap(data.theme)}})
}""",
        {"theme": "state.chart.updateOptions({theme: {mode: state.tMap(data.theme)}})"},
    ),
    (
        """
theme=(state)=>{
    state.chart.updateOptions({theme: {mode: state.tMap(data.theme)}})
}""",
        {"theme": "state.chart.updateOptions({theme: {mode: state.tMap(data.theme)}})"},
    ),
    (
        """
theme=(state, data)=>{
    state.chart.updateOptions({theme: {mode: state.tMap(data.theme)}})
}""",
        {"theme": "state.chart.updateOptions({theme: {mode: state.tMap(data.theme)}})"},
    ),
    (
        """
theme=(state,data)=>{
    state.chart.updateOptions({theme: {mode: state.tMap(data.theme)}})
}""",
        {"theme": "state.chart.updateOptions({theme: {mode: state.tMap(data.theme)}})"},
    ),
    # Nested function definitions
    (
        """
render=()=>{
    color=(x)=>{
        return {"x": x["value"]
    }
}""",
        {
            "render": """\
color=(x)=>{
    return {"x": x["value"]
}"""
        },
    ),
]

EXAMPLE_DIRTY_CLEAN_SCRIPTS = [("  console.log(data)\n", "console.log(data)")]

@pytest.mark.parametrize(["text", "scripts"], EXAMPLE_SCRIPTS)
def test_to_scripts(text, scripts):
    assert text_to_scripts(text) == scripts

@pytest.mark.parametrize(["dirty", "clean"], EXAMPLE_DIRTY_CLEAN_SCRIPTS)
def test_clean_script(dirty, clean):
    assert _clean_script(dirty) == clean

I've tried it on a .js script like below and it work really well. But the hot reloading on .js files would make it even better.

render=()=>{
  state.oMap = function(x){return {"x": x["label"], "y": Math.round(x["score"]*100)}}
  state.theme = function(){return {default: "light", dark: "dark"}[data.theme]}
  state.options=()=>{return {...data._base_options, series: [{name: "Score", "data": Array.from(data.output_json, state.oMap)}], colors: [data.color], theme: {mode: state.theme()}}};
  state.chart = new ApexCharts(plot, state.options());
  state.chart.render();
}
output_json=()=>{
  state.chart.updateOptions(state.options())
}
color=()=>{
  state.chart.updateOptions({colors: [data.color]})
}
theme=()=>{
  state.chart.updateOptions({theme: {mode: state.theme()}})
}
after_layout=()=>{
  window.dispatchEvent(new Event('resize'))
}

philippjfr commented 3 years ago

Agree that it would be nice to support but I'd really want to make it a lot more robust than what you're doing there which is matching on very specific characters and that's difficult.

MarcSkovMadsen commented 1 year ago

I still would very much want something to make the developer experience great.

For example being able to use local css files and js files.

assets/my-component.py

import panel as pn
import param
from panel.reactive import ReactiveHTML

class MyComponent(ReactiveHTML):
    _extension_name = "my-component"

    _template = """<div id="my-component" class="my-component">My Component</div>"""

    __css__ = ["assets/my-component.css"]
    __javascript__ = ["assets/my-component.js"]

    _scripts = {"render": "MyComponent.render()"}

pn.extension("my-component", template="fast")

MyComponent().servable()

assets/my-component.css

.my-component {
    height: 100%;
    width: 100%;
    background: blue;
}

assets/my-component.js

window.MyComponent = {
    render: function () {
        alert("Hello! I am an alert box!!");
    }
}

It sort of works if you put the files in the right folders and add --static-dirs assets=./assets when you serve them. But adding --static-dirs is not what you want to ask users of your custom component to do. And --autoreload for .css and .js files does not work because the files are cached by the tornado handler.

If I could even just point _script='assets/my-component.js and all the functions there would be in the _script dictionary it would be awesome.

Ahh. My editor understands what I am doing

CmpCtrl commented 9 months ago

@MarcSkovMadsen thanks for the example of parsing a .js file to the _scripts dict. I'm close to 900 lines, so it was a nightmare to work on without syntax highlighting. I did have to fiddle with the character matching to get it to work with the auto-formatting from vscode.

MarcSkovMadsen commented 9 months ago

As far as I understand the next version of ReactiveHTML will support external files and many other things. It's called ReactiveESM.

holoviz / panel