crystal-lang / crystal

The Crystal Programming Language
https://crystal-lang.org
Apache License 2.0
19.21k stars 1.61k forks source link

Add `StringLiteral#to_utf16` #14676

Closed ysbaddaden closed 1 week ago

ysbaddaden commented 3 weeks ago

Implements the {{ "hello".to_utf16 }} proposal that merely exposes String#to_utf16 to macros.

Kept as draft until we decide which form we prefer: macro call or macro method.

closes #14670

ysbaddaden commented 3 weeks ago

I didn't run any benchmark. It's probably not so fast because we must transform the Slice(UInt16) into an ArrayLiteral(NumberLiteral) which means lots of allocations.

BlobCodes commented 3 weeks ago

As written in https://github.com/crystal-lang/crystal/issues/14670#issuecomment-2155387689, almost all of the time required to convert a String to UTF-16 in macro land is caused by the parser.

Adding this macro method probably won't be a noticable performance improvement.

ysbaddaden commented 3 weeks ago

@BlobCodes Most likely, yes, but speed ain't my motive. Reusing stdlib means we only implement the algorithm once, and we can have symmetry:

Also, I had fun poking into macro methods: we can return any ASTNode :smiling_imp:

ysbaddaden commented 3 weeks ago

Let's benchmark a bit, generating a utf16 slice literal of "TEST 😐🐙 ±∀ の" 5000 times with this proposal and the proposed macro. the only significant result is the "Semantic (top level)" result (old Intel Haswell i7):

I didn't expect the macro method to be noticeably faster.

edit: and anyway we don't expect to encode 5000 static UTF-16 strings in a program, and even so the impact is barely noticeable compared to everything else (be it the macro or macro method).