Speed it up

[nottrobin]

Look into ways of speeding up the builder. Maybe only update if source files are newer?

[anthonydillon]

Write it in go? :)

On Wed, 19 Oct 2016, 22:10 Robin Winslow, notifications@github.com wrote:

Look into ways of speeding up the builder. Maybe only update if source files are newer?

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/CanonicalLtd/documentation-builder/issues/20, or mute the thread https://github.com/notifications/unsubscribe-auth/ABWRnueoNnkwznQ8F-NV0A1MBAWoRes1ks5q1oc2gaJpZM4KbdZ0 .

[nottrobin]

@anthonydillon patch welcome

[evilnick]

I'm not sure what you added recently, but I noticed that between 0.4.0 and 0.4.3 it is about five times slower. Significantly, it is now about 8 times slower than the old build tool. It might not matter so much on MAAS docs, but for juju docs that would translate into a significant delay

[nottrobin]

@evilnick Yep, this needs addressing. Where does this delay matter most? Is it in building the documentation locally as you're making changes? How long does the Juju documentation currently take to build with the old tool?

[evilnick]

Yeah, the local build is where it kills us - for tricky additions, we may have to build it 4 or 5 times to get the formatting right (tables etc).

For me, the Juju docs currently take 8 seconds to build. I have just now made a rough estimate by plugging all the md files into the maas folder and running documentation-builder there = 78 seconds

[nottrobin]

Okay good to know. The things that take the time are the regex replacements (e.g. here and here) and concatenating the context.yaml files.

My first idea is to compare timestamps between the built .html files (if they exist) and the .md file, and only compile if the .md file is newer. This way, most of the time when building locally (as long as it's not the first time) there should only be one or two files that have changed, which should make the whole run take less than a second.

[evilnick]

Yes regex is a bastard. I was thinking when I have some time to look properly, it might be better all round to make a markdown plugin for rewriting the urls. The benefit is that within the md context, everything is in an element tree, so there is no difficulty finding and changing all the image links for example. As no regex involved it should be ridiculously fast as well. The only downside is that the plugin would have to be instantiated with the path to rewrite to(as I guess this could vary) which may require some structural modifications.

[evilnick]

Oh, also, yes :+1: on comparing timestamps!

[nottrobin]

@evilnick care to check that out if you get a chance? ^