Howdju / howdju

Monorepo for the Howdju crowdsourced fact checking and summarization platform
https://www.howdju.com
GNU Affero General Public License v3.0
5 stars 2 forks source link

Support markdown tables in MediaExcerpts #716

Open carlgieringer opened 2 months ago

carlgieringer commented 2 months ago

E.g.: https://www.howdju.com/media-excerpts/2021

carlgieringer commented 2 months ago

An approach adding just tables might look like:

import { Plugin } from "unified";
import { Node } from "unist";
import { visit } from "unist-util-visit";
import { gfmTable, gfmTableHtml } from "micromark-extension-gfm-table";
import { gfmTableFromMarkdown, gfmTableToMarkdown } from "mdast-util-gfm-table";

const remarkGfmTables: Plugin<[]> = function () {
  const data = this.data();

  add("micromarkExtensions", gfmTable);
  add("fromMarkdownExtensions", gfmTableFromMarkdown);
  add("toMarkdownExtensions", gfmTableToMarkdown);
  add("toHtmlExtensions", gfmTableHtml);

  function add(field: string, value: unknown) {
    const list = (data[field] || (data[field] = [])) as unknown[];
    list.push(value);
  }

  return (tree: Node) => {
    visit(tree, "text", (node: { type: string; value: string }) => {
      if (node.value.includes("|") && node.value.includes("\n")) {
        // This is a simple check for potential table content
        const lines = node.value.split("\n");
        if (lines.length >= 2 && lines[1].replace(/[^|]/g, "").length >= 3) {
          // This looks like it could be a table, so we'll mark it for processing
          node.type = "html";
        }
      }
    });
  };
};

export default remarkGfmTables;
carlgieringer commented 2 months ago

719 adds ~0.14 MB to the bundle size. I'm not sure how much that would reduce if we used the approach above for rendering just tables vs. all of remark-gfm.

Before:

Assets:
  main.css (4.77 MiB)
  premiser-ui.js (1.55 MiB)

WARNING in entrypoint size limit: The following entrypoint(s) combined asset size exceeds the recommended limit (244 KiB). This can impact web performance.
Entrypoints:
  main (6.32 MiB)
      main.css
      premiser-ui.js

After:

Assets:
  main.css (4.77 MiB)
  premiser-ui.js (1.69 MiB)

WARNING in entrypoint size limit: The following entrypoint(s) combined asset size exceeds the recommended limit (244 KiB). This can impact web performance.
Entrypoints:
  main (6.46 MiB)
      main.css
      premiser-ui.js