Incremental rendering

[spanezz]

Ideas for implementing incremental rendering:

• create a rough in-memory filesystem abstraction, with a dict that maps normalised output pathnames (via os.normpath) to core.Page objects. This is needed both to render in-memory for ssite serve and for comparing what should be rendered with the existing contents of web/
• use git log (or GitPython) to see which pages have changed since the last run.
  • Uncommitted pages are always considered modified.
  • A successful rendering operation can be tracked in a git tag.
• jinja2 pages and taxonomy pages need to always be regenerated
• if a findable file has been added or removed from git, trigger a full rebuild, because it can potentially mean that an internal link in any page needs to be rendered differently.
• scan web/ and compare with the relpath that would be generated, to detect which pages need rebuilding because files miss, and what files in web/ need deleting because no pages generate it.

[paternal]

Hello, does this mean that right now, everything is re-generated from scratch?

[spanezz]

Yes, right now everything is re-generated from scratch at each site rebuild. For my use case (around 600 markdown pages), it is currently fast enough not to consider incremental rendering a priority.

[spanezz]

Similar to how markdown caches previous renderings in .staticsite-cache, one can track, with cooperation with jinja2 functions, the dependencies of a page. If the dependencies have not changed, we do not need to rebuild the page

[spanezz]

I forgot to mention: now markdown caches previous markdown→html content in .staticsite-cache, greatly speeding up previous builds. It looks like staticsite is starting to have some incremental rendering.

[spanezz]

This is now working in the refactor branch!

[spanezz]

I merged refactor to master. The links feature cannot do incremental builds yet, but it seems reasonably fast, and not as used as the others.

Finally closing this issue :tada: