Open straight-shoota opened 1 month ago
Like in #14592 there are some spec failures in CI related to iconv. Somebody should investigate that on a macOS machine.
Apparently Apple replaced the system iconv implementation from an old GNU libiconv to a FreeBSD-based one in 2023, so this is why I recalled not seeing those spec failures on my M2 in the past?
The actual bug is somewhat weird, and I have managed to reproduce it in C as well. String#encode
uses an output buffer of 1024 bytes; if a single call to LibC.iconv
fails because the output buffer has some space but is insufficient for the next character, then outbuf_ptr
and outbytesleft
only advance in increments of 32 bytes, even though the correct output bytes are written after outbuf_ptr
for the input bytes it can consume. Those extra output bytes are then lost:
# first call consumes 1023 bytes and outputs 992 bytes; all the `y`s are lost
# second call consumes 3 bytes and outputs 2 bytes for the `好`
("x" * 992 + "y" * 31 + "好").encode("EUC-JP").size # => 994
Updates nixpkgs in
shell.nix
to 24.05 release. This release also brings LLVM 18 (and drops LLVM 11) so we update LLVM as well.Closes #14269, #14592