search

Shopify / liquid-c

Liquid performance extension in C.
MIT License
119 stars 24 forks source link

show full multi-byte UTF8 character in error messages #210

Closed ggmichaelgo closed 9 months ago

ggmichaelgo commented 9 months ago

What are you trying to solve?

With strict parsing mode, Liquid-C is throwing an error message with the first byte of the invalid token. With multi-byte UTF-8 character such as &nbsp, the error message will only display the first byte of the character, \xC2 This error message results the Liquid template output to have invalid UTF-8 byte sequence. 

require 'liquid'
require 'liquid/c'
Liquid::Template.parse("{{ 123 \u00A0 }}", error_mode: :strict)
# Error message:
# Liquid syntax error: Unexpected character \xC2 in "{{ 123   }}" (Liquid::SyntaxError)