Increase performance for building docs

[aedart]

Hi

I have been using your package for some time now - and written lots of documentation with it. It works really great it's simply. However, after the ever increasing amount of pages that I create, I have noticed that it does take more time to build the HTML pages (currently over 700 *.md files to built each time).

Would it be possible for you to change build strategy, so that only "changed" pages are regenerated. Of course, certain elements like menus or indexes might have to be generated regardless of circumstance (I do not know your package that well).

There are many ways to determine if a "page" has changed.

• Checksum of file, stored it inside a local "cache" file
• Git status (assuming git is available on local environment)
• Last modified date of file (If you can trust it from local OS - still has to be stored in some kind of cache)
• ...other?

I know that this is somewhat of a "major" request. But everyone will benefit from it.

[onigoetz]

Wow, 700 pages, that's the biggest documentation I've heard of :)

I can have a look to be able to cache Markdown's compilation results. That might speed up the compilation a lot.

I think a checksum of the files' content is the easiest to make work cross-platform/repository. But I'll experiment and choose the best.

Just know that I don't plan to work on that feature in the coming weeks. So if you are motivated, I accept Pull Requests and would be happy to accept a contribution on that.

Otherwise, thanks for your patience :)

[aedart]

@onigoetz Ty for your quick reply.

If I can manage some time to get into your source-code, then I'll take a look at it. But otherwise, don't worry - I take what I can get, because It's worth the wait. Your package does solve a huge issue at my job... everything else is pure bonus :)

NB: The ~700 pages are due to an internal framework (PHP, mixture of corporate components and adapters for OpenSource). Requires lots of attention, if future generations are going to continue to maintain and develop stuff.

[onigoetz]

So, I added a cache on Markdown content. It divided by two the run time on Daux's documentation.

Can you test it by getting the latest version with composer global require daux/daux.io:dev-master and tell me if it works fine for you ?

I don't know if further optimizations would yield good results. I'll try to look around if I can get some leads.

[aedart]

Hi again. I was about to attempt something similar.

I have, just now, tested and it appears to have had some positive effect. Not much, but it does help. However, there is a minor issue:

Unserialise Cached Payload

You are obtaining the entire cache entry, instead of just the payload. Each of my pages look something like this:

Installation > System Requirements

s:779:"
Table of Contents

    Composer

As a minimum, the following requirements must be meet for this framework to work. Additional requirements might be imposed by concrete components.

    PHP >= 7.1.0
    OpenSSL PHP Extension
    PDO PHP Extension
    Mbstring PHP Extension
    Tokenizer PHP Extension
    XML PHP Extension

Composer

Make sure that you have a Composer installed globally.
"; 

You need to unserialize the payload before using it. :) Btw, is it really need to serialise the content first? Couldn't you just store the entire content without such?

Cache lifetime

How long do the cached files exist inside /tmp (or where ever PHP decides to store them)?

[onigoetz]

I didn't unserialize ? oops :( You are right, I could store them unserialized.

I decided to not manage the lifetime of the cache, because /tmp is automatically cleaned up by the OS from time to time, and it's invalidated automatically relative to the content and configuration that you use. I'll also add the Daux version to the cache key so if some changes are made to the code, it will automatically invalidate the cache.

I will add a daux clear-cache command anyway to be able to do this manually if needed.

[aedart]

Sounds perfect - great job mate :)

[onigoetz]

It's finally released in 0.7.5, I'll have a look if I can improve the performance a bit more

[aedart]

Ty mate