Text processing module architecture

[philcockfield]

Module:

• sys.data.text. <== (text processing)

AST - Abstract Syntax Trees

• MDAST (Markdown)
• HAST (HTML)
• NLCST Natural Language Concrete Syntax Tree format.
  • libraries

Conceptual Naming Conventions

• Processor (a composition of "plugins")

Kinds of String Content ("grammars" of sort)

• MARKDOWN - "Remark". plugin: remark-*

• HTML - "Rehype". plugin: rehype-*

• TEXT - "Retext"
  nlcst (natural language). plugin: retext-english

---

REFS:

• https://unifiedjs.com/learn/guide/using-unified/
• https://unifiedjs.com/learn/guide/create-a-plugin/
• https://unifiedjs.com/learn/guide/syntax-trees-typescript/
• https://unifiedjs.com/learn/recipe/tree-traversal-typescript/
• https://unifiedjs.com/learn/recipe/tree-traversal/
• https://medium.com/unifiedjs/collectively-evolving-through-crowdsourcing-22c359ea95cc
• https://www.ryanfiller.com/blog/remark-and-rehype-plugins/
• https://github.com/unifiedjs/handbook
• https://unifiedjs.com/explore/package/unist-util-is/

---

[philcockfield]

ref

[philcockfield]

[philcockfield]

[philcockfield]

Refs:

• Alan Kay (timestamp)