Pure HTML Runtime Environment

I think there is an opportunity to re-build the VizHub runtime environment to be based on purely HTML standards with ES Modules. In this variant of the runtime, there would be no need to run the source code through any various parsing and processing, meaning this version of the runtime would have zero dependency on Rollup.

Would indirectly solve by use of import maps and native ES modules the following issue:

https://github.com/vizhub-core/vizhub/issues/1001

Here's a start at it:

index.html

<!doctype html>
<html lang="en">
  <head>
    <meta charset="UTF-8" />
    <meta
      name="viewport"
      content="width=device-width, initial-scale=1.0"
    />
    <title>Vanilla Totally</title>
    <style>
      body {
        margin: 0;
        overflow: hidden;
      }
      #viz-container {
        height: 100vh;
      }
    </style>
    <script type="importmap">
      {
        "imports": {
          "d3": "https://cdn.skypack.dev/d3@7.9.0"
        }
      }
    </script>
    <link rel="stylesheet" href="styles.css">
  </head>
  <body>
    <div id="viz-container"></div>
    <script type="module" src="viz.js"></script>
  </body>
</html>

viz.js


import { main } from './index.js';

const container = document.getElementById('viz-container');
let state = {};

const setState = (next) => {
  state = next(state);
  render();
};

const render = () => {
  main(container, { state, setState });
};

render();

index.js

This remains the same contract as the current V3 runtime environment - that index.js exports a function called main that accepts a container, and the { state, setState } state management thing.

import * as d3 from "d3";

export const main = (container) => {
  const width = container.clientWidth;
  const height = container.clientHeight;

  const svg = d3
    .select(container)
    .selectAll("svg")
    .data([null])
    .join("svg")
    .attr("width", width)
    .attr("height", height);

  const data = [
    { x: 100, y: 200, size: 10 },
    { x: 200, y: 150, size: 20 },
    { x: 300, y: 100, size: 15 },
    { x: 400, y: 250, size: 25 },
    { x: 500, y: 300, size: 30 },
  ];

  const xScale = d3.scaleLinear()
    .domain([0, d3.max(data, d => d.x)])
    .range([50, width - 50]);

  const yScale = d3.scaleLinear()
    .domain([0, d3.max(data, d => d.y)])
    .range([height - 50, 50]);

  svg
    .selectAll("circle")
    .data(data)
    .join("circle")
    .attr("cx", d => xScale(d.x))
    .attr("cy", d => yScale(d.y))
    .attr("r", d => d.size)
    .attr("fill", "steelblue");

  const xAxis = d3.axisBottom(xScale);
  const yAxis = d3.axisLeft(yScale);

  svg
    .selectAll("g.x-axis")
    .data([null])
    .join("g")
    .attr("class", "x-axis")
    .attr("transform", `translate(0, ${height - 50})`)
    .call(xAxis);

  svg
    .selectAll("g.y-axis")
    .data([null])
    .join("g")
    .attr("class", "y-axis")
    .attr("transform", `translate(50, 0)`)
    .call(yAxis);

  svg
    .selectAll("text.x-axis-label")
    .data([null])
    .join("text")
    .attr("class", "x-axis-label")
    .attr("x", width / 2)
    .attr("y", height - 20)
    .attr("text-anchor", "middle")
    .text("X Axis Label");

  svg
    .selectAll("text.y-axis-label")
    .data([null])
    .join("text")
    .attr("class", "y-axis-label")
    .attr("x", -height / 2 + 25)
    .attr("y", 30)
    .attr("text-anchor", "middle")
    .attr("transform", "rotate(-90)")
    .text("Y Axis Label");
};

This works great when locally hosted. I suppose this would have the downside of not supporting libraries that don't publish an ESM build.

In order to implement something like this as the next-gen VizHub runtime, I'd want to solve the following before launching it:

[ ] See if we can make it all run in an iFrame that doesn't need to actually make any HTTP requests, so it can run in the VizHub client-side sandbox.
[ ] Investigate the possibility of supporting hot reloading of main, which the current runtime v3 supports and is a really nice (dare I say "killer") feature to have
[ ] Make sure it works with various libraries like TopoJSON and ThreeJS

Some ideas from ChatGPT (o1-preview):

Certainly! Let's update the old implementation to leverage modern browser APIs, focusing on using ES6 modules, fetch, and Blob URLs instead of the older XMLHttpRequest and overriding methods. We'll also use Blob URLs to handle scripts and styles, and we'll intercept fetch requests to serve in-memory files.

Note: While service workers are a powerful tool for intercepting network requests, they cannot be registered from Blob URLs or data URLs due to security restrictions. Since we want to avoid calling out to a server and keep everything client-side, we'll use Blob URLs and override the fetch API in a controlled manner.

Here's the updated code:

export default function (template, files) {
  // Collect the files, excluding 'index.html' and 'thumbnail.png'
  const fileMap = {};
  Object.keys(files).forEach((fileName) => {
    if (fileName === 'index.html' || fileName === 'thumbnail.png') return;
    fileMap[fileName] = files[fileName].content;
  });

  // Create Blob URLs for each file
  const blobUrlMap = {};
  Object.keys(fileMap).forEach((fileName) => {
    const content = fileMap[fileName];
    const blob = new Blob([content], { type: getContentType(fileName) });
    const blobUrl = URL.createObjectURL(blob);
    blobUrlMap[fileName] = blobUrl;
  });

  // Replace references in the template
  template = replaceReferences(template, blobUrlMap);

  // Inject code to override 'window.fetch' for fetch requests to the original filenames
  const fetchOverrideScript = `
    <script type="module">
      (function() {
        const originalFetch = window.fetch;
        const blobUrlMap = ${JSON.stringify(blobUrlMap)};
        window.fetch = function(input, init) {
          const url = typeof input === 'string' ? input : input.url;
          const pathname = new URL(url, location.href).pathname.slice(1);
          if (blobUrlMap[pathname]) {
            return originalFetch(blobUrlMap[pathname], init);
          }
          return originalFetch(input, init);
        };
      })();
    </script>
  `;

  // Prepend the fetch override script to the template
  template = fetchOverrideScript + template;

  return template;

  function getContentType(fileName) {
    if (fileName.endsWith('.js')) {
      return 'application/javascript';
    } else if (fileName.endsWith('.css')) {
      return 'text/css';
    } else if (fileName.endsWith('.html')) {
      return 'text/html';
    } else if (fileName.endsWith('.json')) {
      return 'application/json';
    } else if (fileName.endsWith('.svg')) {
      return 'image/svg+xml';
    } else if (fileName.endsWith('.png')) {
      return 'image/png';
    } else if (fileName.endsWith('.jpg') || fileName.endsWith('.jpeg')) {
      return 'image/jpeg';
    } else if (fileName.endsWith('.gif')) {
      return 'image/gif';
    } else {
      return 'text/plain';
    }
  }

  function replaceReferences(html, blobUrlMap) {
    // Replace <script src="file.js" type="module"> with Blob URL
    html = html.replace(/<script\s+([^>]*?)src=["']([^"']+)["']([^>]*)><\/script>/g, (match, beforeSrc, src, afterSrc) => {
      const fileName = src.startsWith('./') ? src.slice(2) : src;
      if (blobUrlMap[fileName]) {
        const attrs = `${beforeSrc.trim()} ${afterSrc.trim()}`.trim();
        return `<script type="module" src="${blobUrlMap[fileName]}" ${attrs}></script>`;
      }
      return match;
    });

    // Replace <link href="file.css" rel="stylesheet"> with Blob URL
    html = html.replace(/<link\s+([^>]*?)href=["']([^"']+)["']([^>]*)>/g, (match, beforeHref, href, afterHref) => {
      const fileName = href.startsWith('./') ? href.slice(2) : href;
      if (blobUrlMap[fileName]) {
        const attrs = `${beforeHref.trim()} ${afterHref.trim()}`.trim();
        return `<link href="${blobUrlMap[fileName]}" ${attrs}>`;
      }
      return match;
    });

    return html;
  }
}

Explanation:

Collecting Files:
- We iterate over the provided files object and collect all files except index.html and thumbnail.png.
- We store the contents in a fileMap for easy access.
Creating Blob URLs:
- For each file in fileMap, we create a Blob with the appropriate MIME type.
- We generate a Blob URL using URL.createObjectURL(blob) and store it in blobUrlMap.
Replacing References in the Template:
- We scan the template HTML string for <script> and <link> tags that reference our files.
- We replace the src or href attributes with the corresponding Blob URLs.
- We ensure that scripts have type="module" as per the requirement.
Overriding the Fetch API:
- We inject a script at the beginning of the template that overrides window.fetch.
- The overridden fetch checks if the requested URL matches one of our files.
- If it does, it redirects the request to the Blob URL.
- This allows any fetch calls in the user code to access the in-memory files without network requests.
Returning the Modified Template:
- The function returns the modified HTML template with the injected scripts and updated references.

Leveraging Modern Browser APIs:

ES6 Modules: We support scripts with type="module" and ensure that module scripts are properly loaded from Blob URLs.
Blob URLs: We use Blob URLs to create object URLs for in-memory file contents, allowing us to reference scripts and styles without external servers.
Fetch API: Instead of overriding XMLHttpRequest, we intercept fetch requests, which is the modern way of handling HTTP requests in the browser.

Usage Example:

Suppose you have the following files object:

const files = {
  'index.html': {
    content: `
      <html>
        <head>
          <link rel="stylesheet" href="styles.css">
        </head>
        <body>
          <h1>Hello World</h1>
          <script type="module" src="script.js"></script>
        </body>
      </html>
    `,
  },
  'script.js': {
    content: `
      import { message } from './module.js';
      console.log(message);
    `,
  },
  'module.js': {
    content: `
      export const message = 'This is a module!';
    `,
  },
  'styles.css': {
    content: `
      body { background-color: lightblue; }
    `,
  },
};

You can use the updated function as follows:

const parseCode = /* the updated function */;
const modifiedTemplate = parseCode(files['index.html'].content, files);

// Now you can set the `modifiedTemplate` as the `srcdoc` of an iframe
const iframe = document.createElement('iframe');
iframe.srcdoc = modifiedTemplate;
document.body.appendChild(iframe);

Considerations:

Security Restrictions: Service workers cannot be registered from Blob URLs or data URLs due to security policies. Using Blob URLs and intercepting fetch requests client-side allows us to work within these constraints.
Browser Support: The approach uses modern browser features like ES6 modules and Blob URLs, which are supported in most modern browsers.
No External Dependencies: The entire runtime environment is created in the browser without calling out to a server to build or fetch files.
Limitations: Overriding window.fetch should be done cautiously, as it can affect other network requests. In this controlled environment, it's acceptable, but it's important to be aware of potential side effects.

Benefits:

Live Editing and Running: Users can edit and run code directly in the browser, with immediate feedback.
Modular Code Support: By supporting scripts with type="module", users can write modular JavaScript code using the latest syntax.
Simplified Environment: Eliminates the need for server-side processing or file serving, simplifying the development and execution environment.

This updated implementation modernizes the original code by leveraging current web technologies, providing a seamless, in-browser coding experience that aligns with modern development practices.

vizhub-core / vizhub

Pure HTML Runtime Environment #1042