Emoji in messages synced from main device render as question marks

[daveriedstra]

First of all, thanks, scli is fantastic.

So: emoji in messages show as question marks and it seems like only if they're synced from the main device (my phone in my case). In this image you can see sent and received messages with emoji affected by this (the sent message I sent from my phone), and also an emoji in the compose area (which demonstrates that it's not a font or a terminal issue). Sending emoji in messages and reactions from scli works and they continue to render fine, so my guess is that it must be something to do with how they are synced.

possibly related to #33

[exquo]

Thanks for reporting!

Odd, seems to be working for me. Could you try running scli with --debug and post the log file (in ~/.local/share/scli/log) with the synced / received messages containing the emojis?

[daveriedstra]

Thanks for having a look at this. I'm not using the history feature, so here's the log for an emoji sent to myself (with the sensitive info removed, hope that's okay)

INFO:root:scli 0.7.1
DEBUG:root:signal-cli account: linked device
DEBUG:root:callf: `['signal-cli', '-u', '[redacted]', '--output=json', 'daemon']`
INFO:root:daemon_log: INFO  DaemonCommand - Starting daemon in single-account mode for [redacted]
INFO:root:daemon_log: INFO  DaemonCommand - DBus daemon running on SESSION bus: org.asamk.Signal
INFO:root:signal-cli dbus service started
INFO:root:signal-cli 0.10.4.2
DEBUG:root:Daemon: json_data = 
{'account': '[redacted]',
 'envelope': {'source': '[redacted]',
              'sourceDevice': 1,
              'sourceName': '[redacted]',
              'sourceNumber': '[redacted]',
              'sourceUuid': '4db[...]f2b',
              'syncMessage': {'sentMessage': {'destination': '[redacted]',
                                              'destinationNumber': '[redacted]',
                                              'destinationUuid': '4db[...]f2b',
                                              'expiresInSeconds': 0,
                                              'message': '?',
                                              'timestamp': 1649616092941,
                                              'viewOnce': False}},
              'timestamp': 1649616092941}}
INFO:root:Setting status = `sending` for timestamp = 1649616092941
INFO:root:Setting status = `sent` for timestamp = 1649616092941

[exquo]

Based on the line

                                              'message': '?',

the issue is with decoding signal-cli's output, not with the rendering in the terminal.

Your system's locale needs to support UTF-8. Check that the $LANG variable's value ends in .UTF-8.

Might be related: https://github.com/AsamK/signal-cli/issues/352, https://github.com/AsamK/signal-cli/issues/457

Is the emoji printed normally if you run

signal-cli -u +123YourPhoneNumber890 --output=json deamon

and receive the same emoji?

[daveriedstra]

That makes sense, I guess that makes it a signal-cli issue more than an scli issue. $LANG is en_CA.UTF-8 and the json output from signal-cli does not render the emoji correctly. I'll move my attention to fixing it in signal-cli -- thanks for the help!

[daveriedstra]

for time travellers: seems to be fixed with an update to signal-cli 0.10.5