Only root diet template is cached.

[Stretto]

Only root templates get cached. If my root template is very simple and composed of several other templates then caching is ineffective if any of those change.

It would be nice if each dt file is cached instead during each execution of the include statement.

The current design also makes caching less effective because included templates may change while the parent doesn't. Currently it seems the caching feature doesn't monitor file changes. I end up with 2 cached files but I have about 10 dt files. The two cached files(index.dt and error.dt) essentially do nothing but include the other dt files.

I suggest that in the cached file both the filename, creation time stamp, and file size be included in a meta block(comment at the start would work). These are compared during compilation to determine consistency of the cached files. It is done also at the include level.

Ultimately, a real time caching and synchronization design would be nice. Basically add a real time file monitor for changes and then recompile the changes on the file and re include them. This doesn't seem like it would be that difficult but would require some type of "dll" like way to add and remove the template code.

E.g., if you wrapped all d code in a diet template in a function and used function pointers to point to them(each of the templates in a nested design) and called them in the proper sequence then it should be rather simple to add and remove recompiled templates(handling current connections using the old templates might pose security/usability issue though). (the execution tree would correspond/be derived from to the "include tree")

Then just a way to recompile the templates... shouldn't be too hard since dmd would be required to be available.

This technique scales well with more complex web site designs. It allows one to edit without having to recompile everything.

[s-ludwig]

Caching each template separately is unfortunately not generally possible, because the compilation step is separated into distinct parsing, (possibly user defined) AST transformation, and code generation phases. The code generation phase has, in general, no way to know how a piece of the input AST relates to an input file, apart from that some of the nodes come from a particular file (other nodes might have been generated in the transformation phase).

However, I consider the template caching mechanism a temporary workaround for DMD's slow CTFE engine anyway. Once the new CTFE engine that is currently in the works is finished, the template compilation time should get faster by a factor of at least 10, which renders any caching mechanism more or less useless.

The planned way to implement real-time recompilation is by adding a file monitoring functionality to DUB, which would trigger a re-compile of the whole project once a file changes. Compiling individual templates into separate shared libraries breaks down with the template alias based API that is used to pass symbols into the template's scope. It would, however, work for a layered API that works using normal function parameters of some sort, so this may still be worth looking at, if performance remains an issue.

[schveiguy]

Not sure if this was the case back then, but the current version hashes all dependencies, so I think this is already covered.

Perhaps the OP meant to write "hashed" instead of "cached"? I mean, the dependency FILES aren't cached, but modifying a dependency will re-compile the file, which is as effective as you need.