feature suggestion | Synchronous/Asynchronous Helpers

[robincsamuel]

Registering a helper as Sync or Async, It helps to listen for callbacks, and get the data from callback.

[kpdecker]

This has come up in the past but we didn't act on it as the use case wasn't clear. Since handlebars still has to wait until all of the data is available to render providing async evaluation is simply a convience for something that the code that generates the context can do in a much more efficient manner.

Basically adding async evaluation at this point would be quite expensive both in terms of compatibility and runtime performance that I don't quite see a use case for right now. Do you have a concrete example of what you are trying to do?

[robincsamuel]

I agree with the performance issues, but I think, it will be better, if we can optionally make it aysnc, for example, RegisterHelper & RegisterHelperAsync, or anything like that.

Actually, I came to think about these async handlebars, while i work with node.js. I am working on some applications using express.js & the template engine i use is handlebars. So, If i need to get some value from a database call, while compiling the view, its not possible with this synchronous working,

For example,

Handlebars.registerHelper('getDbValue', function(id) {
     var Model = require('./myModel.js');
     Model.getValue(id, function(data){
           return data;
     });
});

The above example will not work, and returns nothing. The following is my concept. And, I don't know, if it is totally right or it can be implemented or not. Just using a callback function instead of return, in case of async method.

Handlebars.registerHelperAsync('getDbValue', function(id, callback) {
     var Model = require('./myModel.js');
     Model.getValue(id, function(data){
           callback(data);
           //or
           //callback(new Handlebars.SafeString(data)); //in case of safestring.
     });
});

I am experiencing more issues like the above one, and I can show more example according to my scenario, if anyone is interested in this feature.

Thanks

[jwilm]

@robincsamuel, When including database lookups in view generation, the whole idea of MVC separation goes at the window. I think you are arguing that you might not know you need the data until the view is being rendered, but to me that suggests functionality that should be implemented at the controller level rather than when generating your view. Combined with the performance considerations @kpdecker mentioned, async helpers just seem wrong. -- my 2c

[robincsamuel]

I just used that example to convey my problem. And I am not trying to argue, but, just made a suggestion. I hope, it will help if we call a function with callback from the helper. Anyway, thanks for your time :) @kpdecker @jwilm

[kpdecker]

At this point the stance of the project is that data resolution should be done prior to calling the template. Outside of the business logic in or outside of the template concerns, async resolution is more of utility behavior that other libraries such as async are much better suited at handling.

[tomasdev]

I want to comment something. Even though it would be throwing out the window the DB example, this could be really helpful for mutational templates. For example, a template with "subviews" inside, and that you don't want to split into several other templates. I just want to update a part in the view, and have a simple logic for that, instead of either repainting my whole view (flickering effect) or having my controller to build the whole thing for all these "mini views"

What do you think?

[robincsamuel]

@tomasdev I mean that :)

[ErisDS]

This feature is made available in node with express if you use https://github.com/barc/express-hbs. However the async version of helpers do not play nicely with subexpressions and a few other edge cases.

I would like to see this feature reconsidered for inclusion in handlebars, or at least consider how handlebars core can better support this kind of extension.

I believe Ghost demonstrates a clear and valid (although perhaps uncommon) usecase for async helpers, because our view layer is customisable.

On the frontend, all of Ghost's templates are provided by the theme. The theme is a very thin layer of handlebars, CSS and client JS, the only data it has access to is that which we provide in advance. It does not have access to a controller or any behaviour changing logic. This is very deliberate.

In order to expand out the theme API, we want to start adding helpers which define additional data collections the theme would like to make use of. For example something like:

{{#fetch tags}}
.. do something with the list of tags..
{{else}}
No tags available
{{/fetch}}

Ghost has a JSON API which is available both internally and externally. So this fetch query would map to our browse tags function. It doesn't need to use ajax/http to all the endpoint, instead, an async helper can fetch this data from the API internally and carry on as usual.

I don't argue that this is a common use case, and I accept it breaks the standard MVC model, but I believe it is valid and useful.

[robincsamuel]

@ErisDS Great news! And me too don't argue that its common issue, but it helps.

[ErisDS]

It's worth noting in this case, that many of the operations we are currently using async helpers for are synchronous under the hood, but they're structured as promises.

To give a detailed example...

All data in Ghost is accessed via an internal API. This includes global info like settings. Requests to the settings API hit a pre-populated in-memory cache before hitting the database, so we're actually just returning a variable, but by structuring this as a promise it's easy to go off to the database if we need to.

It also ensures everything is consistent - otherwise the settings API would be synchronous, and all the other internal data requests would be async, which would make no sense.

I know that structuring everything with promises can be quite confusing at first, but it is one of those things that you don't understand how you lived without once you've got it. With generators coming in ES6, support for asynchronous resolution of functions will baked right in to JavaScript - and this similar issue: https://github.com/wycats/handlebars.js/issues/141 mentions that it would be nice to make handlebars work with yield.

I'm not sure how the coming release of HTMLbars might impact on this, but I think it at least warrants further discussion.

[rikkertkoppes]

Ran into another use case while trying to create a helper for acl resolution. This would fit nicely into my templates:

        {{#allowedTo 'edit' '/config'}}
            <li>
                <a href="/config">Config</a>
            </li>
        {{/allowedTo}}

But the actual isAllowed method from node-acl is asynchronous (allowing a database backend for instance).

A workaround is to fetch all the user permissions beforehand (allowedPermissions), but that's kinda itchy

[ErisDS]

@kpdecker Any further thoughts on these use cases?

[kpdecker]

@ErisDS I understand the desire here but I severely doubt this will ever make it into the language in callback or promises form. This is something that is very hard to do cleanly from an API perspective and effectively requires us to rewrite large portions of the template engine to support it. My recommendation is that all of this be handled before the rendering cycle is entered by the upstream model/data source.

The yield idea is an interesting one but if someone wanted to take a look at what would be required there, that would be an amazing research project, but browser support for that seems very far off to me and I honestly haven't messed around with any of those features yet on any of my projects.

[guiprav]

Just my "two" (well, couple) cents you might want to consider:

• MVC is not sacrosanct. Things aren't wrong simply because they appear to contradict MVC. One has to evaluate whether the alternatives don't provide net positive benefits over following MVC strictly.
• If the view asks the controller for data, not models directly, it's not a violation of the MVC anyway, is it?
• It could perhaps be argued that having the controller know beforehand everything the view will need is a duplication of information (i.e. the information "X, Y, Z, W are necessary" is duplicated in views and controllers.) In other words, our current practice may be a violation of the DRY principle, which is much more important than MVC, imo.
• The performance hit of asynchronous helpers for the purposes of loading only the models necessary to the views being rendered might easily be compensated by loading less data from the database.

[ghost]

I might be able to offer a better example where it would be usefull to have.

We work a lot with cordova for mobile applications and need to localize for many languages. Cordova offers functions to help with formatting dates, numbers, currencies and so on. The problem is that they all require a async callback.

Example:

Handlebars.registerHelper('stringToNumber', function(string, type)
{
    type = type || 'decimal';
    navigator.globalization.stringToNumber(string, function(number)
    {
        return number;
    }, function()
    {
        return NaN;
    }, {
        type: type
    });
});

This would be awesome to have imo.

[nknapp]

I have found the handlebars-async package on npm. But it's a little older and I don't know if it works with the current Handlebars-version.

I have also just written something similar for promises. The package promised-handlebars allows you to return promises from within helpers. I plan to use it in one of my projects, but so far it hasn't been used in a production environment. But there are unit tests for several few edge-cases (such as calling async-helpers from within async block-helpers) and they are all green...

[ErisDS]

@nknapp that sounds amazing! express-hbs has async support, and the async works for block helpers, but nesting async helpers does not work - so I'm really interested to see this working - means there's hope yet for express-hbs :+1:

[nknapp]

@ErisDS, do you think I should post it there. I didn't now that express-hbs couldn't nest async helper. My primary focus is not express, but a README-generator I'm currently working on. I would really appreciate other people to try it (promised-handlebars) give feedback.

[ElChupacabra26]

To add to the valid use use-cases, what if you need to fetch values from a translation DB based on current locale?

<div class="howItWorks">
    {{{i18nFetch id=how-it-works locale=locale}}}
</div>

Moreover, what about adding a CMS block from a DB entry using a dynamic id like:

<div class="searchCms">
    {{{cmsLoader 'search-{$term}' term=params.input defaultId='search-default'}}}
</div>

This is especially useful for server-side rendering (i.e. using express-handlebars).

[dwhieb]

Here's another use case: I'm writing a documentation-generator for Swagger (simple-swagger), which allows for external schema definitions. I'd like to write a Handlebars helper that recognizes when a schema is defined externally, goes to the provided URL where that schema lives, retrieves it, and uses that data to render that portion of the Handlebars template. If I had to retrieve this data before calling Handlebars' compile method, I would have to recursively iterate through a JSON document whose structure I don't know in advance, find all the instances of external schemas, retrieve them, and insert them into the JSON.

Basically, anytime a Handlebars template is used to render JSON schema data (json-schema.org), an async render method would be useful, because JSON schema always allows subparts of a schema to be defined externally.

[nknapp]

@dwhieb have you had a look at bootprint-swagger for the documentation-generator? It's almost what you describe (except that external schemas are not yet implemented, but that would be a great feature). If you have any feedback, please open an issue there.

And, I think promised-handlebars works quite well with async helpers.

[richardkazuomiller]

I have a use case where being able to use promises in helpers would be useful. I'm using handlebars to generate the HTML for my blog. In order to build valid structured data for each article, I need to get the dimensions I'm using for the article image. Right now, I'm doing it like this:

{{#imageSize post.frontMatter.previewImage}}
  <div itemprop="image" itemscope itemtype="https://schema.org/ImageObject">
    <meta itemprop="url" content="{{#staticResource ../post.frontMatter.previewImage}}{{/staticResource}}">
    <meta itemprop="width" content="{{width}}">
    <meta itemprop="height" content="{{height}}">
  </div>
{{/imageSize}}

The imageSize helper works because it reads the file synchronously, but ideally it should be able to do it asynchronously so rendering pages isn't slowed down by I/O. Also, doing this for an image at a URL, instead of on the filesystem, is impossible.

I'll look into using promised-handlebars and express-hbs but I think the ability to use promises in helper functions would be a great addition to Handlebars!

[guiprav]

FWIW, I've been doing a lot of asynchronous HTML rendering using Hyperscript, hyperscript-helpers, ES7's async/await, and it's been a real joy. But of course, that solution only works for HTML. An async solution with Handlebars would let us generate other kinds of files asynchronously... However for HTML I think I'm never looking back!

[darrenoakey]

going to give express-hbs a try - but I think in 2018 its a strange thing not to support. I know that there is a purist view that async stuff should not be done as part of the view but instead in some magical thing called a controller (as if MVC is somehow unarguably "correct") - but that's a little short sighted for two key reasons

a) external libraries - most people now are writing entirely with async/await - I think in my code more than 9 out of 10 functions are async... in some cases "just in case". Not supporting an async function means all async libraries are suddenly completely inaccessible

b) generic controller functions. I would argue that something like this:

    {{#query "select name, total from summary"}}
          <tr><td>{{this.name}}</td><td>{{this.total}}</td></tr>
    {{/query}}

is shorter, cleaner, easier to maintain, and basically superior in every conceivable way when compared with having a bespoke controller function that sticks that stuff into a variable and passes it into a template, which the template then has to know about and access.

[Atulin]

I see nothing's changed for the last 8 years? With everything and their mother getting full async support, even Node slowly adding top-level await, Hbs still offers no support for async helpers?

[nknapp]

@Atulin Honestly, I don't think this is going to happen soon, too. It may seem like a big project, but from 2016 to 2020, I was basically the only one who did active maintenance. Now, @jaylinski has taken over. I wanted to support im, but in the last months, I had other things to do in my free time... So he is more or less alone in this.

What I want to say: But there is no such thing as a "active development team" or any kind of "planning". All we have done in the last years is answer to questions, fix bugs, especially security issues.

Unless this changes, there will be no native async support for Handlebars.

[Atulin]

Gotcha. I found a package that adds async helper support and it's working well, so I'll say "good enough"

[nknapp]

Do you mean this? https://www.npmjs.com/package/promised-handlebars

[Atulin]

https://github.com/gastonrobledo/handlebars-async-helpers

I did check the one you linked, but it hasn't been updated for half a decade, and the examples use this weird q package

[CheyenneForbes]

@Atulin if I was to submit a pull request based on some of the ideas in @gastonrobledo's project, is there a chance of you or @jaylinski accepting/merging it?

[Atulin]

I'm not a maintainer of Hbs, so it's not me you should be asking lol

[jaylinski]

@CheyenneForbes yes, there is a chance that we will accept an async-helpers-improvement (but only in version 5).

[toverux]

This should very much be supported. You won't need it 90% of the time sure, but you desperately need it when you do. In our use case we allow clients to create templates in our platform. We provide an helper allowing to load an image from a library of images. We have to preload the whole library prior to calling handlebars, making it very inefficient. We had to finally use handlebars-async-helpers (* careful, it has some bugs) because we had a new use case : allow to manipulate images and rotate them with a helper (images arrive as base64 into handlebars, not URLs). This is not doable synchronously and we couldn't rotate the whole library in advance because this would take minutes. Now we have an async helper that does it on-demand. Unfortunately, handlebars-async-helpers treats async tasks sequentially but at least we've been able to implement it. We're actually thinking to move away from handlebars if this is not implemented, because this raises serious issues for uses cases like ours.

[nknapp]

@Atulin About https://www.npmjs.com/package/promised-handlebars:

I know, there hasn't been an update for over 6 years. And I am not actively working on it anymore. On the other hand, it has almost no dependencies and no known security issues as well.

And it's a small library with a tight use-case so there is actually no need for an update imho.

Handlebars has not had a real update for years as well. When there is a major version bump I will make sure that promised-handlebars stays compatible.

If anyone wants to make a PR updating the documentation to remove q... Go for it.

I am not promising fast responses here. But if there is something critical, Handlebars maintainers can reach me through faster channels.

In case an actively maintained version for promised-handlebars is worth to you so much that you would be willing to pay for it...

[nknapp]

In any case. Of course it would be more powerful to have it included in Handlebars directly. But I don't see the resources for implementing this.

[yosiasz]

if performance is the issue why not throttle it using required pagination and limit the data returned?