UI broken: lines not lining up

[jeanlucthumm]

Having the following issue

This is similar to #48 but I can't find emojis in my contacts. Unfortunately I couldn't think of a way to screenshot the contacts without violating their privacy but here are all non alphanumeric characters I could find:

) and (
~
] and [
+

There is one interesting thing though, previously where there was only a number now it seems like it's guessing the name of the contact somehow, but it's replacing white space with a ? symbol as follows:

+11231231234 ~John?Deere

Could it be that the ? symbol is the culprit?

[g33kex]

I can reproduce, I have those non alphanumeric characters:

) and (
] and [
+
é
-

It also seems to happen when the terminal window is too small and names have to wrap on the second line.

[exquo]

Thanks for reporting!

So far I could not reproduce the problem: when contacts' names contain those symbols, or when the names are wrapped over multiple lines, the boxes' lines are not broken..

Tried contact names (by editing signal-cli data file) like

John Doe (
John - Doe
John [Doe
John Doé

Tested on:

• gnome-terminal, xterm, urxvt

• urwid 2.1.2 (latest from PyPI)

  $ python3 -c 'import urwid; print(urwid.__version__)'
  2.1.2

• DejaVuSansMono font

  $ fc-match Monospace
  DejaVuSansMono.ttf: "DejaVu Sans Mono" "Book"

• scli (master or v0.4.0)

Does this happen only when those symbols are in contacts' names, or in messages' text also?

Might be related: https://github.com/urwid/urwid/wiki/FAQ#why-are-box-borders-not-displayed-correctly-on-the-linux-vconsole

---

it seems like it's guessing the name of the contact somehow

The string after ~ is the contact's "profile name" that they have entered when registering their phone number with signal. It is shown in scli if there is no corresponding contact in signal-cli's contactStore (i.e. they are not in your phone's contacts).

The profile name by itself is not a problem as long as it does not contain special symbols (emojis, etc). The question mark might be a sign that it does, and the terminal does not render it correctly. You can check what that symbol is supposed to be in signal-desktop (or mobile) next to the contact's name; also, it would be in ~/.local/share/signal-cli/data/+YOUR_PHONE_NUM file in profileStore -> profiles -> <the contact's entry> -> profile -> name.

---

a way to screenshot the contacts without violating their privacy

You can blur / mask the contacts' names on the screenshot using an image editing programs (e.g. GIMP); see scli's own screenshot.

[exquo]

This might be an issue at the intersection of urwid, terminal emulator and the font it uses.

Here is a short snippet to check if the very basic code would mess up the line:

#!/usr/bin/env python3
import urwid
text_w = [urwid.Text(l) for l in (
    "No special chars",
    "( - ]",
    "Special chars with text ( ) [ ] - +",
    )]
listbox_w = urwid.ListBox(text_w)
box_w = urwid.LineBox(listbox_w)
urwid.MainLoop(box_w).run()

Could you try saving it (as e.g. 115.py) and running with

python3 115.py

You can change the text in the double quotes, add or remove them, to see if the lines of the box get broken.

[g33kex]

Thanks so much for your answer!

I tested on: alacritty, kitty, gnome terminal, with various fonts and font sizes, and it's always the same problem. It happens when browsing the contact view without even needing to open a conversation, so it's for sure the text in the contact names only that's causing the issue, not in the messages.

I'm using the scli-git package from the AUR, version r242.09abe5b-1

I tried your snippet, and the bug is not present, all lines are drawn correctly, even when I add the name of all my contacts with the symbols.

Note that I'm running on Arch Linux with Wayland/sway, but that shouldn't matter I believe.

The bug doesn't not happen in a TTY, lines are drawn correctly.

[jeanlucthumm]

I'm also using scli-git from AUR.

I ran the example you gave and the lines seem to be fine:

How do you get to the signal-cli data file? I'd like to inspect whatever character that ? is on my end

[g33kex]

Just to be sure it's not an issue with the AUR package, I tested directly by cloning master and the issue persists. I have urwid version 2.1.1 and signal-cli version 0.8.1

I'm also interested in getting to the signal-cli data file to try to remove the contacts one by one to see which one is the culprit.

[jeanlucthumm]

Something interesting I just found: For the contacts that have the border shifted one to the left, if I renamed them to (seemingly) the same thing using :renameContact, the border fixed itself. The particular contact for me was

Lea Irodita

So you can see there weren't any emoji or even any special symbols. Note that this contact was not previously renamed by me, so I'm not sure where it got the original name from.

Also interesting is that this is the case for every contact for which the border was shifted - they were all automatically named somehow (different from the +123123123 ~First Last, it's actually just First Last). I wonder if these were somehow pulled from the config of the signal desktop app?

Edit: Oh I just remembered this comment:

It is shown in scli if there is no corresponding contact in signal-cli's contactStore (i.e. they are not in your phone's contacts).

So I'm assuming these were contacts found in signal-cli's contactStore.

[exquo]

Thank you guys for the updates!

The signal-cli's data file is in ~/.local/share/signal-cli/data/+PHONENUM. The contacts list is in the contactStore section of that file. The contacts' names there are obtained from the master device (e.g. your phone) that signal-cli has been linked with.

Based on @jeanlucthumm's last comment, it sounds like there are some special symbols in the contact's names, even though they don't look special. Maybe somewhere between the pulling of the contacts data from the signal servers and saving it in signal-cli's data file, the encoding for e.g. "space" symbol changes to something that looks like space (most of the time), but is interpreted by urwid as e.g. "double space" (if there's such a thing).

I've added to the code snippet above to pull the text from the signal-cli's contacts: https://gist.github.com/exquo/450f6b9988cbc053dbf2151b1da94625 Could you please try running it and see if it causes the issue?

[g33kex]

Alright I think I found the problem.

The issue is still present when fetching directly the contacts with your snippet, so I analyzed the strings of the contact names: they all have a junk Unicode character before and after them. All the contact names seem to start with the character 0x2068 and end with the character 0x2069. I have no idea where those come from...

[exquo]

Nice dig!

Would be interesting to know what's causing it. My guess - those contacts' names are already modified in the phone's storage, before signal gets involved. They might have been obtained from a cloud backup, shared through e.g. email and imported from a vcf, or edited with another contacts app.. Many possibilities to corrupt a record here.

If you feel like probing further, you can extract the contacts from the phone (exported as VCF files; 'shared' through any number of apps, including signal itself; or the whole contacts database can be copied and queried) and see if they already have extraneous characters, or some other unexpected features. This will not tell you where those extra chars came from of course, only confirm that it's not the signal app nor signal-cli that are doing the damage.

Either way though, looks like it's an upstream issue. We can mark it as an instance of #48, taking a broad interpretation of 'emoji', namely: any char that is "non-trivial" (for the terminal emulator). In an unlikely case that signal-cli turns out to add those extra chars, we can report the issue there.

As a workaround, to avoid broken lines in the UI, contacts can be renamed in scli with :renameContact as @jeanlucthumm mentioned, or on the phone and then synced up.

[vargn]

I'm getting this issue also. Every contact in my list is showing this broken line. Except my own number (note to self) and a group (v2). I tried renaming every contact with :renameContact, and I renamed them to the same name. And that fixed the broken line. But within an hour it was broken again.

I used to store my contacts in the cloud on my google account. And this maybe could be the reason why these unicode characters showed up. So i migrated my contacts to local storage on my phone. I deleted and added some contacts. I also did a re-link just for good measure, but the issue is still persistent.

When I tried to rename on the phone, it won't work. I even googled the instructions for how to do it. But in conversation settings in Signal, when i tap on the contact name. Nothing happens. Which is really confusing. Couldn't find anything about this on google.

[cycneuramus]

I'm having this issue as well. The UI lines are misaligned for every named contact, but not for groups or phone numbers. My contacts are synced from Nextcloud, as they have been for years, but it's only recently that I've been seeing this issue in scli.

[exquo]

it's only recently that I've been seeing this issue in scli

I don't think anything relevant changed in scli itself, the non-printable chars should have affected the past versions as well, assuming those utf chars had been there. So maybe this is signal-cli bug after all. Could you guys please verify that you also have them in your signal-cli's contactStore? Just run the updated troubleshooting script - it will print the utf codes for unprintable chars.

I'll make a workaround to remove those extraneous / padding chars in scli.

I tried renaming every contact with :renameContact, and I renamed them to the same name. And that fixed the broken line. But within an hour it was broken again.

Because of #119, after getting a 'sync' message from the master device (phone), the renamed contacts are reverted back to their names on it. So my advice above about renaming contacts locally won't work. And if it's a signal-cli bug, renaming contacts on the phone won't help either.

[AckslD]

I'm also having the same issue (in particular with contacts that only are shown as phone numbers, i.e. with +):

[cycneuramus]

it's only recently that I've been seeing this issue in scli

I don't think anything relevant changed in scli itself, the non-printable chars should have affected the past versions as well, assuming those utf chars had been there. So maybe this is signal-cli bug after all. Could you guys please verify that you also have them in your signal-cli's contactStore? Just run the updated troubleshooting script - it will print the utf codes for unprintable chars.

I'll make a workaround to remove those extraneous / padding chars in scli, if everyone is getting same ones.

I tried renaming every contact with :renameContact, and I renamed them to the same name. And that fixed the broken line. But within an hour it was broken again.

Because of #119, after getting a 'sync' message from the master device (phone), the renamed contacts are reverted back to their names on it. So my advice above about renaming contacts locally won't work. And if it's a signal-cli bug, renaming contacts on the phone won't help either.

I ran the script, and it seems I have the same extraneours chars as @g33kex, i. e. the contact names all start with \u2068 and end with \u2069.

[exquo]

Thank you guys for the feedback! I have opened an issue upstream, and will push a scli-side fix shortly.

[cycneuramus]

That fixes it for me. Thanks a bunch!

[cycneuramus]

A minor issue: on receiving a new message, the bottom bar now says New message from '\u2068<contact name>\u2069

[AckslD]

The recent changes seem to have fixed things in most cases but I still get a weird layout in group chats, see right-hand side of picture below (the one border which is off between the contacts and messages is due to emoji in the group name). Does not seem to happen in individual chats:

[exquo]

Looks like stripping those chars needed to be done more thoroughly. Hopefully now fixed for the rest of the widgets.

[AckslD]

Thanks @exquo! Things look good now also in the group chats :)