Feature: support U+1F600..U+1F64F Unicode characters (emoticons)

[jackBzzz]

TransGUI 5.18

Need support for Unicode emoticon characters in torrent names, file names and paths. For example, character "😠" is not being shown correctly in the TransGUI interface. As a result, one can't open a file containing such a character directly from TransGUI.

[lighterowl]

Possibly related to #1378

[lighterowl]

Emojis are outside the BMP so the JSON Transmission sends us will use two UTF-16 code units to represent those. Thus this is most likely caused by https://gitlab.com/freepascal.org/fpc/source/-/issues/38624 and should be fixed if you compile the project with FPC 3.3.1.

[fredo-47]

Where do you get FPC 3.3.1 ? It seems to me that 3.2.2 is their latest version.

[lighterowl]

3.2.2 is the last stable version, 3.3.1 is the latest development version.

Binaries seem hard to be hard to come by indeed, the only viable option is to compile from source. I'll see if I can set things up in my own fork.

[lighterowl]

tl;dr broken in Transmission itself, tested with 3.0.0 and 4.0.1.

Unfortunately, it appears that handling this kind of characters is broken in Transmission itself - at least in version 3.0 that I'm currently testing with.

I created a torrent from the following contents with mktorrent :

$ tree 🤔
🤔
└── 😇.txt

1 directory, 1 file

$ mktorrent -o think.torrent 🤔
mktorrent 1.1 (c) 2007, 2009 Emil Renner Berthing

hashed 1 of 1 pieces.
writing metainfo file... done

The resulting torrent file is :

$ xxd think.torrent 
00000000: 6431 303a 6372 6561 7465 6420 6279 3133  d10:created by13
00000010: 3a6d 6b74 6f72 7265 6e74 2031 2e31 3133  :mktorrent 1.113
00000020: 3a63 7265 6174 696f 6e20 6461 7465 6931  :creation datei1
00000030: 3637 3736 3037 3336 3465 343a 696e 666f  677607364e4:info
00000040: 6435 3a66 696c 6573 6c64 363a 6c65 6e67  d5:filesld6:leng
00000050: 7468 6934 6534 3a70 6174 686c 383a f09f  thi4e4:pathl8:..
00000060: 9887 2e74 7874 6565 6534 3a6e 616d 6534  ...txteee4:name4
00000070: 3af0 9fa4 9431 323a 7069 6563 6520 6c65  :....12:piece le
00000080: 6e67 7468 6933 3237 3638 6536 3a70 6965  ngthi32768e6:pie
00000090: 6365 7332 303a dca5 1952 447d 80bd 35bf  ces20:...RD}..5.
000000a0: 631b c21f 0664 8798 b7e0 6565            c....d....ee

This is on Linux, so UTF-8 is used for encoding the characters. As expected, there's f0 9f 98 87 at offset 0x5E, and f0 9f a4 94 at offset 0x71.

However, the name of the torrent is garbled in Transmission's torrent-add reply already even when using transmission-remote, and thus appears garbled in transgui as well.

$ transmission-remote 192.168.1.2:9091 -a think.torrent -b
posting:
--------
{"arguments":{"metainfo":"ZDEwOmNyZWF0ZWQgYnkxMzpta3RvcnJlbnQgMS4xMTM6Y3JlYXRpb24gZGF0ZWkxNjc3NjA3MzY0ZTQ6aW5mb2Q1OmZpbGVzbGQ2Omxlbmd0aGk0ZTQ6cGF0aGw4OvCfmIcudHh0ZWVlNDpuYW1lNDrwn6SUMTI6cGllY2UgbGVuZ3RoaTMyNzY4ZTY6cGllY2VzMjA63KUZUkR9gL01v2Mbwh8GZIeYt+BlZQ=="},"method":"torrent-add","tag":10}

-- cut --

got response (len 145):
--------
{"arguments":{"torrent-added":{"hashString":"544140c2ab05cb38c07d1a1420b6d214aa544ed8","id":2065,"name":"\u1f914"}},"result":"success","tag":10}

1f914 is the correct code point for "Thinking Face Emoji", but characters are supposed to be saved using UTF-16 when using the \u JSON representation. Thus this string should be \ud83e\udd14.

[lighterowl]

Fix pending in Transmission : https://github.com/transmission/transmission/pull/5096

[lighterowl]

Testing with current Transmission HEAD (https://github.com/transmission/transmission/commit/848212eea11da1a989734c2b09d8ac13f5a8030f) shows that no changes in transgui are needed :

The transgui version I'm testing was compiled with fpc 3.2.2 which might work for this particular case but will probably break with more complex ones due to the already mentioned https://gitlab.com/freepascal.org/fpc/source/-/issues/38624 .

[lighterowl]

Transmission 4.0.2 was just released with the fix included, which means this should be fixed if you're running that server version and transgui compiled with fpc 3.2.3 or newer.