Extraneous trailing lines/chars in the output of iconv applet

[d-faure]

Under busybox version:

BusyBox v1.37.0-FRP-5181-g5c1a3b00e (2023-08-18 12:18:42 BST)
(mingw64-gcc 12.2.1-8.fc38; mingw64-crt 10.0.0-4.fc38; glob; Unicode)

While converting a regular utf-8 (with BOM) text file to UTF-16LE, I had extraneous trailing lines/chars in the output of iconv applet. The command was:

iconv -t UTF-16LE -f UTF-8 -o file_utf16le.txt file_utf8.txt

The same conversion made with an external iconv tool result in the same number of line

[rmyorston]

I tried the demo file UTF-8-demo.txt, to which I added a BOM and converted from Unix to DOS format. The output from busybox-w32 iconv was identical to that from the glibc iconv on Linux.

Can you provide an example of a file that doesn't work for you?

[d-faure]

My source file is: https://github.com/d-faure/Stardock-Software-Localization/blob/main/Start-Products/Start11%20v2/fr.lng

Here's my shell output:

$ iconv --help
BusyBox v1.37.0-FRP-5181-g5c1a3b00e (2023-08-18 12:18:42 BST)

Usage: iconv [-lc] [-o outfile] [-f from-enc] [-t to-enc] [FILE]...

Convert text between character encodings

    -l  List all known character encodings
    -c  Silently discard characters that cannot be converted
    -o  Use outfile for output
    -f  Use from-enc for input characters
    -t  Use to-enc for output characters

$ iconv -t 'UTF-16LE' -f 'UTF-8' fr.lng > fr-utf16le.lng

$ tail -n 1 fr.lng
"Autohide with a top aligned taskbar requires enhanced taskbar enabled with the autohide option provided by Start11"="Le masquage automatique combiné à l'alignement en haut nécessite l'amélioration des barres et le masquage de Start11"

$ tail -n 1 fr-utf16le.lng
la barre des tÔches

Note:

• The last line (947th) of fr.lng is an empty one.
• The iconv binary of w64devkit [BusyBox v1.37.0.git (2023-08-01 12:16:57 UTC)] is having the same behavior
• I had to rely on iconv from https://github.com/mlocati/gettext-iconv-windows to get a proper file

[rmyorston]

OK, I see what the problem is. As usual it's all my fault.

Should be fixed in the latest prerelease.