arnodel / golua

A Lua compiler / runtime in Go
Apache License 2.0
86 stars 10 forks source link

Lua 5.4 unicode support #61

Closed arnodel closed 2 years ago

arnodel commented 2 years ago

Lua 5.4 introduces support for non-standard unicode code points, including utf8 support for them. Quoting from the docs:

The UTF-8 encoding of a Unicode character can be inserted in a literal string with the escape sequence \u{XXX} (with mandatory enclosing braces), where XXX is a sequence of one or more hexadecimal digits representing the character code point. This code point can be any value less than 231. (Lua uses the original UTF-8 specification here, which is not restricted to valid Unicode code points.)

The above is implemented in this PR

Also the utf8 decoding functions acquire a "lax" mode allowing decoding any value encoded as above. See the section on UTF-8 in the Lua docs, or the tests in this PR.

codecov[bot] commented 2 years ago

Codecov Report

Merging #61 (7a148d9) into lua5.4 (de8f01a) will increase coverage by 0.01%. The diff coverage is 90.00%.

:exclamation: Current head 7a148d9 differs from pull request most recent head 9a48d8a. Consider uploading reports for the commit 9a48d8a to get more accurate results Impacted file tree graph

@@            Coverage Diff             @@
##           lua5.4      #61      +/-   ##
==========================================
+ Coverage   87.87%   87.89%   +0.01%     
==========================================
  Files          93       95       +2     
  Lines       10227    10376     +149     
==========================================
+ Hits         8987     9120     +133     
- Misses        952      968      +16     
  Partials      288      288              
Impacted Files Coverage Δ
runtime/gocont.go 87.13% <0.00%> (-5.42%) :arrow_down:
luastrings/utf8.go 93.93% <93.93%> (ø)
lib/utf8lib/utf8lib.go 100.00% <100.00%> (ø)
luastrings/escape.go 100.00% <100.00%> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more Δ = absolute <relative> (impact), ø = not affected, ? = missing data Powered by Codecov. Last update de8f01a...9a48d8a. Read the comment docs.

arnodel commented 2 years ago

https://github.com/arnodel/golua-tests/blob/5514d9875c4b08df9c72b6a4e4b0ad801d95d178/literals.lua#L75 seems to be failing

Yeah, this is a backward incompatible change. This error is normal as it would error in Lua 5.3 but is now valid in 5.4. Sorry I forgot to mention that in the PR description.