jupyter-book / mystmd

Command line tools for working with MyST Markdown.
https://mystmd.org/guide
MIT License
192 stars 60 forks source link

Allow site build to complete quickly then re-render as slower steps complete #1351

Open fwkoch opened 2 months ago

fwkoch commented 2 months ago

This issue is motivated by recent problems with slow DOI loading. In those cases, after running myst start everything just seemed to freeze for a long time before the built site was available. There are a few other slow steps in the build process, e.g. image processing. It would improve the user's experience if the build finished faster, leaving placeholder content for the slower steps, then, on completion of those steps, re-rendered.

This would take a little refactoring of the processMdast/postProcessMdast steps - e.g. transforms might need to check if prerequisites have been met, rather than assuming all the previous steps are complete; slow transforms will need to be async with a callback to re-render. We would also need to be careful about designing the placeholder content so the user understands the build has not yet finished. However, I think this would make the local authoring experience smoother, and I have a feeling it would also improve consistency of the transform code, forcing us to rely on fewer assumptions about the current state of the mdast when a transform runs.

agoose77 commented 2 months ago

I think this ties into more explicit dependency management, which we touched on somewhere else. This would be useful for plugins to be able to inject themselves into arbitrary points in the transform pipeline.