search

terascope / teraslice

Scalable data processing pipelines in JavaScript
https://terascope.github.io/teraslice/
Apache License 2.0
50 stars 13 forks source link

Update docusaurus #3686

Closed busma13 closed 1 month ago

busma13 commented 1 month ago

This PR makes the following changes:

godber commented 1 month ago

Ok, since I rushed you along a little bit, what things have been deferred?

busma13 commented 1 month ago
sotojn commented 1 month ago

Major problem with compiling the website. yarn docs in the top level will build documentation files with a syntax that isn't compatible with the new docusaurus version. We need to look into how ts-scripts docs operates and change it to create files with the right syntax.

Error when building docs: (This is done by running yarn install then yarn build in the website website directory)

[ERROR] Error while parsing Markdown front matter.
This can happen if you use special characters in front matter values (try using double quotes around that value).
[ERROR] Loading of version failed for version current

[ERROR] Error: Unable to build website for locale en.
    at tryToBuildLocale (/Users/jsoto/Workspace/teraslice/website/node_modules/@docusaurus/core/lib/commands/build.js:54:19)
    at async /Users/jsoto/Workspace/teraslice/website/node_modules/@docusaurus/core/lib/commands/build.js:65:9
    at async mapAsyncSequential (/Users/jsoto/Workspace/teraslice/website/node_modules/@docusaurus/utils/lib/jsUtils.js:20:24)
    at async Command.build (/Users/jsoto/Workspace/teraslice/website/node_modules/@docusaurus/core/lib/commands/build.js:63:5) {
  [cause]: Error: Can't process doc metadata for doc at path path=/Users/jsoto/Workspace/teraslice/docs/packages/data-mate/api/enums/column_aggregations.KeyAggregation.md in version name=current
      at processDocMetadata (/Users/jsoto/Workspace/teraslice/website/node_modules/@docusaurus/plugin-content-docs/lib/docs.js:146:15)
      at async Promise.all (index 77)
      at async doLoadVersion (/Users/jsoto/Workspace/teraslice/website/node_modules/@docusaurus/plugin-content-docs/lib/index.js:121:34)
      at async loadVersion (/Users/jsoto/Workspace/teraslice/website/node_modules/@docusaurus/plugin-content-docs/lib/index.js:162:28)
      at async Promise.all (index 0)
      at async Object.loadContent (/Users/jsoto/Workspace/teraslice/website/node_modules/@docusaurus/plugin-content-docs/lib/index.js:170:33)
      at async /Users/jsoto/Workspace/teraslice/website/node_modules/@docusaurus/core/lib/server/plugins/plugins.js:38:23
      at async Promise.all (index 0)
      at async /Users/jsoto/Workspace/teraslice/website/node_modules/@docusaurus/core/lib/server/plugins/plugins.js:139:25
      at async loadSite (/Users/jsoto/Workspace/teraslice/website/node_modules/@docusaurus/core/lib/server/site.js:135:45) {
    [cause]: YAMLException: incomplete explicit mapping pair; a key node is missed; or followed by a non-tabulated empty line at line 2, column 17:
        title: Data-Mate: `Column Aggregations Key Aggre ... 
                        ^
        at generateError (/Users/jsoto/Workspace/teraslice/website/node_modules/gray-matter/node_modules/js-yaml/lib/js-yaml/loader.js:167:10)
        at throwError (/Users/jsoto/Workspace/teraslice/website/node_modules/gray-matter/node_modules/js-yaml/lib/js-yaml/loader.js:173:9)
        at readBlockMapping (/Users/jsoto/Workspace/teraslice/website/node_modules/gray-matter/node_modules/js-yaml/lib/js-yaml/loader.js:1027:9)
        at composeNode (/Users/jsoto/Workspace/teraslice/website/node_modules/gray-matter/node_modules/js-yaml/lib/js-yaml/loader.js:1359:12)
        at readDocument (/Users/jsoto/Workspace/teraslice/website/node_modules/gray-matter/node_modules/js-yaml/lib/js-yaml/loader.js:1525:3)
        at loadDocuments (/Users/jsoto/Workspace/teraslice/website/node_modules/gray-matter/node_modules/js-yaml/lib/js-yaml/loader.js:1588:5)
        at load (/Users/jsoto/Workspace/teraslice/website/node_modules/gray-matter/node_modules/js-yaml/lib/js-yaml/loader.js:1614:19)
        at Object.safeLoad (/Users/jsoto/Workspace/teraslice/website/node_modules/gray-matter/node_modules/js-yaml/lib/js-yaml/loader.js:1637:10)
        at module.exports (/Users/jsoto/Workspace/teraslice/website/node_modules/gray-matter/lib/parse.js:12:17)
        at parseMatter (/Users/jsoto/Workspace/teraslice/website/node_modules/gray-matter/index.js:109:17) {
      reason: 'incomplete explicit mapping pair; a key node is missed; or followed by a non-tabulated empty line',
      mark: Mark {
        name: null,
        buffer: '\n' +
          'title: Data-Mate: `Column Aggregations Key Aggregation`\n' +
          'sidebar_label: Column Aggregations Key Aggregation\n' +
          '\x00',
        position: 17,
        line: 1,
        column: 16
      }
    }
  }
}
[INFO] Docusaurus version: 3.4.0
Node version: v18.16.0
error Command failed with exit code 1.
sotojn commented 1 month ago

Also running yarn docs in the top level of teraslice throws this error:

★  star      Building docs for e2e
ts-scripts docs [packages..]

Update documentation for all packages or a specific package

Positionals:
  packages  Run scripts for one or more packages                                                                                                                                                [array] [default: []]

Options:
  -h, --help     Show help                                                                                                                                                                                  [boolean]
  -v, --version  Show version number                                                                                                                                                                        [boolean]

Error: Unknown pkg, [object Object]
    at /Users/jsoto/Workspace/teraslice/packages/scripts/dist/src/helpers/doc-builder/sidebar.js:51:23
    at Array.map (<anonymous>)
    at processList (/Users/jsoto/Workspace/teraslice/packages/scripts/dist/src/helpers/doc-builder/sidebar.js:38:35)
    at updateSidebarJSON (/Users/jsoto/Workspace/teraslice/packages/scripts/dist/src/helpers/doc-builder/sidebar.js:76:48)
    at async buildPackages (/Users/jsoto/Workspace/teraslice/packages/scripts/dist/src/helpers/doc-builder/index.js:26:5)
error Command failed with exit code 1.
busma13 commented 1 month ago

There are a few formatting options that we can't use until we upgrade to typedoc-plugins-markdown v4.1.0. That requires typedoc v0.26.0 or greater, which has a change that currently prevents typescipt from building.