search

com-lihaoyi / mill

Mill is a fast JVM build tool that supports Java, Scala and Kotlin. 2-4x faster than Gradle and 4-10x faster than Maven for common workflows, Mill aims to make your project’s build process performant, maintainable, and flexible
https://mill-build.org/
MIT License
2.24k stars 358 forks source link

Speed up Mill Docsite generation #3411

Open lihaoyi opened 3 months ago

lihaoyi commented 3 months ago

time ./mill -i docs.fastPages takes 1-2 minutes now on my laptop for incremental updates. This is unacceptable for something as visual a documentation. A large portion of this appears to be the graphviz diagram generation, but that is unavoidable as we want diagrams in our documentation. The core problem seems that our Antora setup doesn't seem to be caching and re-using any of the processed asciidoc files

We should figure out how to make Antora faster, or replace Antora entirely with something else that is faster.

lihaoyi commented 3 months ago

Also the asciidoctor-kroki uses a remote server to render the graphviz diagrams, which apart from being terribly slow is also unstable and returns 504s sometimes

[18:12:36.543] WARN (asciidoctor):
    file: /Users/lihaoyi/Github/mill/out/docs/devAntoraSources.dest/modules/ROOT/pages/Cross_Builds.adoc
    source: /Users/lihaoyi/Github/mill (branch: docs-overhaul <worktree> | start path: out/docs/devAntoraSources.dest)
    msg: {
      "err": {
        "package": "asciidoctor-kroki",
        "message": "Skipping graphviz block.",
        "stack": "Error: Skipping graphviz block.\n    at wrapError (/Users/lihaoyi/Github/mill/out/docs/npmBase.dest/node_modules/asciidoctor-kroki/src/asciidoctor-kroki.js:48:22)\n    at constructor.<anonymous> (/Users/lihaoyi/Github/mill/out/docs/npmBase.dest/node_modules/asciidoctor-kroki/src/asciidoctor-kroki.js:186:30)\n    at Object.apply (/Users/lihaoyi/Github/mill/out/docs/npmBase.dest/node_modules/@antora/asciidoc-loader/node_modules/@asciidoctor/core/dist/node/asciidoctor.js:26192:21)\n    at constructor.$$instance_exec (/Users/lihaoyi/Github/mill/out/docs/npmBase.dest/node_modules/asciidoctor-opal-runtime/src/opal.js:3846:24)\n    at Opal.send (/Users/lihaoyi/Github/mill/out/docs/npmBase.dest/node_modules/asciidoctor-opal-runtime/src/opal.js:1671:19)\n    at Proxy.$$19 (/Users/lihaoyi/Github/mill/out/docs/npmBase.dest/node_modules/@antora/asciidoc-loader/node_modules/@asciidoctor/core/dist/node/asciidoctor.js:19974:24)\n    at Opal.send (/Users/lihaoyi/Github/mill/out/docs/npmBase.dest/node_modules/asciidoctor-opal-runtime/src/opal.js:1671:19)\n    at constructor.$$process (/Users/lihaoyi/Github/mill/out/docs/npmBase.dest/node_modules/@antora/asciidoc-loader/node_modules/@asciidoctor/core/dist/node/asciidoctor.js:19978:20)\n    at constructor.$$call (/Users/lihaoyi/Github/mill/out/docs/npmBase.dest/node_modules/asciidoctor-opal-runtime/src/opal.js:17898:26)\n    at Opal.send (/Users/lihaoyi/Github/mill/out/docs/npmBase.dest/node_modules/asciidoctor-opal-runtime/src/opal.js:1671:19)\nCaused by: Error: GET https://kroki.io/graphviz/svg/eNqN0bsOwiAUBuDdpzhxtwHUEVcXJx1NY0BOC5GIARprjO9urYmXiOIEnPx8h4sytRcHDXM4DwC82O-U8Xyx7BZ7pxDWQYsDculaOBoVNSeg0dQ6dpMQTxZ5ZaxFBbdh66zz_KhNxLIDQiPv-NY2IaLfsL4JgBUSLR9Wzq1ZQVk57Kt3TomgUfWFZ6AITVWZ9pa7pGCahGkOplmYJGGSg0kWnrzDUvjfJ34ECmnq1W97mrRZzmb_2OOkTXI2SdifjwWj2dcdn3_2mqZf0iyZfr_p5QoAaOhq - server returns 504 status code; response: \n    at httpRequest (/Users/lihaoyi/Github/mill/out/docs/npmBase.dest/node_modules/asciidoctor-kroki/src/http/http-client.js:45:9)\n    at Object.httpGet [as get] (/Users/lihaoyi/Github/mill/out/docs/npmBase.dest/node_modules/asciidoctor-kroki/src/http/http-client.js:53:10)\n    at Object.httpGet [as get] (/Users/lihaoyi/Github/mill/out/docs/npmBase.dest/node_modules/asciidoctor-kroki/src/http/node-http.js:5:65)\n    at KrokiClient.getImage (/Users/lihaoyi/Github/mill/out/docs/npmBase.dest/node_modules/asciidoctor-kroki/src/kroki-client.js:70:30)\n    at module.exports.save (/Users/lihaoyi/Github/mill/out/docs/npmBase.dest/node_modules/asciidoctor-kroki/src/fetch.js:47:78)\n    at createImageSrc (/Users/lihaoyi/Github/mill/out/docs/npmBase.dest/node_modules/asciidoctor-kroki/src/asciidoctor-kroki.js:67:38)\n    at processKroki (/Users/lihaoyi/Github/mill/out/docs/npmBase.dest/node_modules/asciidoctor-kroki/src/asciidoctor-kroki.js:163:25)\n    at constructor.<anonymous> (/Users/lihaoyi/Github/mill/out/docs/npmBase.dest/node_modules/asciidoctor-kroki/src/asciidoctor-kroki.js:184:16)\n    at Object.apply (/Users/lihaoyi/Github/mill/out/docs/npmBase.dest/node_modules/@antora/asciidoc-loader/node_modules/@asciidoctor/core/dist/node/asciidoctor.js:26192:21)\n    at constructor.$$instance_exec (/Users/lihaoyi/Github/mill/out/docs/npmBase.dest/node_modules/asciidoctor-opal-runtime/src/opal.js:3846:24)"
      }
    }

I didn't manage to find any other graphviz diagram plugin for antora, but maybe i didnt look hard enough. https://docs.asciidoctor.org/diagram-extension/latest/ seems to be ruby-only and antora uses asciidoctor-js