Unicode astral symbols are not normalized by default

[dotnetCarpenter]

Before filing issues, please check the following points first:

• [ ] Please don't open issues for security issues. Instead, file a report at https://www.npmjs.com/advisories/report?package=handlebars
• [ ] Have a look at https://github.com/wycats/handlebars.js/blob/master/CONTRIBUTING.md
• [ ] Read the FAQ at https://github.com/wycats/handlebars.js/blob/master/FAQ.md
• [X] Use the jsfiddle-template at https://jsfiddle.net/4nbwjaqz/4/ to reproduce problems or bugs

This is actual a doc issue, since the behaviour that Handlebars.js has now is also what I would want. However I just spend 2 hours trying to figure out why emojis could would not be rendered by Handlerbars.js but worked fine everywhere else.

This fiddle illustrates the issue: https://jsfiddle.net/dotnetCarpenter/bzvdejfx/2

We have an utf-8 string like Iñtërnâtiônàlizætiøn☃💩, that we want to output. But anything other that ASCII is garbled by Handlebars.js. It's a very easy fix though. Just wrap it in new Handlebars.SafeString(this) which seems to be what Handlebars.js does automagically if you pass a value through a function.

The only issue with this is, that it's MAGIC! Can the documentation please be a little more clear. Perhaps with a emoji example? It does not have to be poo. 💣 is also nice ;)

[dotnetCarpenter]

Everything you want to know about JavaScript and astral symbols + too much more, Mathias Bynens wrote about here https://mathiasbynens.be/notes/javascript-unicode

[nknapp]

Wrapping everything in a safestring opens the doors for cross-site-scripting attacks. We should look into this. I see no reason why Unicode chars shouldn't be supposed to work.

[nknapp]

You should not pass a string as parameter to the template.

template('Iñtërnâtiônàlizætiøn☃💩')

Using the template {{0}} with a string as input will only return the first character, because it translates to (input["0"]).

In your example, you are also using the function calls to "identity", "safestring" and "normalize" like they are not supposed to be used, but accidentally, you get the results you expected. I will explain below...

Still I would like to verify that we get the same outputs of your fiddle: In my browser (Version 77.0.3865.90 (Official Build) (64-bit, on Linux)), your example (slightly modified at https://jsfiddle.net/91nsyk6t/7/) looks like this:

I have modified your example (at https://jsfiddle.net/91nsyk6t/8/) to use the template

<p>Naked: {{unicodeWord}}</p>

with the template call

template({unicodeWord: 'Iñtërnâtiônàlizætiøn☃💩'});

the I get the whole word:

---

About safestring, identity etc...

You are using the syntax: {{#identty}}{{0}}{{/identity}} to call the identity-helper. Usually, when you call a helper like this (i.e. as a block-helper), you are need to call options.fn() to evaluate the inner part (docs). The way you do it, will completely ignore the inside of the block and just plainly return the current context, which is your word (https://jsfiddle.net/91nsyk6t/12/).

---

Conclusion: As far as I can see, this has nothing to do with unicode astral-planes. For which I am relieved, because this would have been a nasty research to find the bug.

But thank you for referencing that interesting blog-post.

[dotnetCarpenter]

My screenshots of https://jsfiddle.net/91nsyk6t/7/

OS: Linux Mint 19.2 Tina

Mozilla Firefox 70.0.1

Chromium 78.0.3904.70 Built on Ubuntu , running on LinuxMint 19.2

[dotnetCarpenter]

So what I can gather is that I'm suppose to use block-helpers as:

Handlebars.registerHelper('identity', identity)
Handlebars.registerHelper('safeString', safeString)
Handlebars.registerHelper('normalize', normalize)

document.getElementById('output').innerHTML = template({t:'Iñtërnâtiônàlizætiøn☃💩'});

function identity (opt) { return opt.fn(this) }
function safeString (opt) { return new Handlebars.SafeString(opt.fn(this)) }
function normalize (opt) { return String(opt.fn(this)).normalize('NFC') }

---

My real use-case is render a table from a nested array.

[["1","💣"],["1","1"]]

But I can not figure out how to write the template to do that and the documentation suggest creating a new helper.

So I modified my data model to have a named property (row):

const data = [
    {
        "row": [
            "1",
            "💣"
        ]
    },
    {
        "row": [
            "1",
            "1"
        ]
    }
]
function emoji () { return this }
Handlebars.registerHelper('emoji', emoji)

<table>
    {{#each board}}<tr>

      {{#each row}}
        <td>{{#emoji}}{{0}}{{/emoji}}</td>
      {{/each}}

    </tr>{{/each}}
  </table>

This works but I have to pass the string value through a function as discussed before.

[nknapp]

This purpose of the documentation you mention is to give examples on how to implement block-helpers. Maybe this should be made clearer.

The most straight forward way to access the current context directly, is {{this}} (https://jsfiddle.net/5udksr18/1/) or {{.}} (https://jsfiddle.net/5udksr18/).

A cleaner (more readable) way to write your template would be using block-params (https://handlebars-draft.knappi.org/guide/block-helpers.html#block-parameters): https://jsfiddle.net/5udksr18/3/

[dotnetCarpenter]

I definitely think there should be an example that renders primitive values and not only objects. Also I can not find any documentation about .

Also I am confused about this. If it the same as context as explained in Basic Blocks?

Handlebars.registerHelper("noop", function(options) {
  return options.fn(this);
});

But then later in Simple Iterators the first argument is context - should it be options?

Handlebars.registerHelper("each", function(context, options) {
  var ret = "";

  for (var i = 0, j = context.length; i < j; i++) {
    ret = ret + options.fn(context[i]);
  }

  return ret;
});

Would you accept a PR for adding your last example https://jsfiddle.net/5udksr18/1/ to https://handlebars-draft.knappi.org/guide/block-helpers.html#block-parameters?

[nknapp]

Context

"Context" refers to the current evaluaton context. The main template starts with the root-object, but the context changes as you call block helpers (like #with or #each) or call partials. I think this is best explained with a small fiddle: https://jsfiddle.net/hmbz46qn/5/

• The this of the helper function is the same as {{this}} outside the block.
• {{this}} inside the block is the context passed as a parameter to options.fn(...)
• The object passed to options.fn(...) can be completely independent of the current this.
• options is always the last parameter of a helper. If you want to call the helper with {{#block-helper param1 param2}} then you need to do Handlebars.registerHelper(function (param1, param2, options) { ... }).

Adding the example

What we call block parameters is the syntax using as |...| like in {{#each . as | row | }}. It is not what I did in https://jsfiddle.net/5udksr18/1/. I would recommend the syntax shown in https://jsfiddle.net/5udksr18/3/, because it is clearer and omits some other problems (like in #1300).

What I think is really missing is a section about {{this}} and {{.}} in https://handlebars-draft.knappi.org/guide/expressions.html

If you want to add a section there, go ahead. Finding the right place is the difficult this, I guess.

[dotnetCarpenter]

updated

Hmm I had some trouble getting block parameters to work but then I removed { strict: true } from Handlebars.compile and it worked.

The script description does not make me understand the issue.

strict: Run in strict mode. In this mode, templates will throw rather than silently ignore missing fields. This has the side effect of disabling inverse operations such as {{^foo}}{{/foo}} unless fields are explicitly included in the source object.

Perhaps it's a bug? https://jsfiddle.net/dotnetCarpenter/w2pu81dL/

[nknapp]

This is a bug (as far as I can tell..., it is the same see #1459). I haven't had a look why this is the case, but it does not have anything to do with unicode symbols, luckily.

I'll close this in favour of #1459