SailfishOS port

[Kabouik]

I was trying to package havoc for Sailfish OS, and was surprised I could actually build it on SFOS 3.3.0.16 with this (for reference to remember which packages were necessary)

git clone https://github.com/ii8/havoc.git
cd havoc
devel-su
zypper install wayland-protocols-devel
zypper install libxkbcommon-devel
zypper install wayland-devel
exit
make CFLAGS="-O2 -DNDEBUG"
devel-su make install

I now have the proper binary in /usr/local/bin, unfortunately I get this when trying to execute it:

[nemo@Sailfish havoc]$ havoc
could not open font file: No such file or directory
using fallback font
your compositor does not support xdg_wm_base, make sure you have the latest version

I am not a coder at all, and am not familiar with how compositors work (or even what they are exactly), but SailfishOS does use Wayland. This is the version we have: 1.17.0+git1-1.4.1.jolla.

Is there any hope, or does havoc depend on a more recent version? Another SFOS user told me this:

<Nico[m]> Sailfish doesn't have a concept of maximize, minimize, etc, so wm_base doesn't make much sense. You can probably rip that out and it may work fine

Would that be doable? What should I look for in the sources?

Alternatively, we have some partial Flatpak support on SFOS (just in case you would be planning on a Flatpak release in the future).

[ii8]

In wayland there are clients(such as havoc and all the other applications that want to display windows and similar things) and a compositor which is responsible for putting all the windows together onto the screen and sending input events to the clients etc.

xdg_wm_base is what's called an interface in Wayland lingo, an agreed way to communicate between client and server. xdg_wm_base is a way for a client to ask a compositor to show a wl_surface as a window. https://wayland.freedesktop.org/docs/html/apa.html#protocol-spec-wl_surface

It looks like Sailfish OS uses a compositor called "Lipstick", if lipstick does not have xdg_wm_base we need some other way to display the terminal window. I can't seem to find any docs where it says what interfaces are supported.

https://github.com/ii8/havoc/blob/507446c92ed7bf8380a58c5ba2b14aba5cdf412c/main.c#L1263-L1282 This registry_get function is called for every interface that the compositor supports, if you add a

printf("%s %d\n", i, version);

statement in this function(just after line 1265 for example) it will print all the interfaces the compositor supports along with the version.

Then we can look if there is any suitable interface we can use instead of xdg_wm_base to display our window.

[Kabouik]

Thanks for your very detailed answer, it's greatly appreciated.

Here are the interfaces I could get using your instructions:

[nemo@Sailfish havoc]$ havoc
could not open font file: No such file or directory
using fallback font
wl_compositor 3
wl_data_device_manager 1
wl_shm 1
wl_output 2
qt_hardware_integration 1
android_wlegl 2
qt_output_extension 1
qt_surface_extension 2
qt_touch_extension 1
qt_windowmanager 1
wl_seat 2
wl_shell 1
lipstick_recorder_manager 1
alien_manager 1
your compositor does not support xdg_wm_base, make sure you have the latest version

Note that Sailfish OS is a mobile OS, so as mentioned in my original post, it may just no concept of minimize/maximize/etc. (although it does have a multitask view that shows thumbnails of running apps), so perhaps instructing havoc to run in fullscreen might help?

[ii8]

Fullscreen or not makes no difference to havoc, everything is rendered the same.

It looks like lipstick still uses wl_shell which is deprecated in favor of xdg_wm_base, so the correct solution here would be to change lipstick.

Alternatively, if that's not an option, I would be happy to have a separate branch where wl_shell is used instead of xdg_wm_base. It should be relatively easy to change, calls on xdg wm_base/surface objects need to be changed to their respective wl shell/surface equivalents, and the toplevel and surface configuration code merged into one function. At least that's all it looks like on first glance.

[ii8]

Also, don't let what that Nico[m] person said confuse you, it makes no sense and is irrelevant.

[Kabouik]

That's an encouraging answer, thanks. A Lipstick upgrade from Jolla (the company that develops SFOS) seems unlikely to me, but I may be wrong. In any case, it's good to hear that you wouldn't mind a second branch for this specific case. I admit I wouldn't really know where to start if I wanted to submit a PR though, despite your instructions.

It might be worth mentioning that, in the end, Lipstick may just be a layer to QtWayland. See this post by one of the original authors: https://github.com/bugaevc/wl-clipboard/issues/85#issuecomment-634799562

[ii8]

QtWayland certainly does support the xdg shell, so lipstick just needs to enable it somehow I imagine. This should at least be tried first I think, since it's the correct solution.

My compositor doesn't actually support wl_shell(because it's deprecated) so it'll be a bit tricky to implement it. I can try to write it as long as you can do the testing for me. I could also try to help you and walk you through implementing it yourself if you want.

[Kabouik]

Rburchell mentioned that Jolla do not always follow upstream very closely for some packages though, and I don't know where to look to check their version of QtWayland.

I am of course happy to do the testing for havoc, no problem. I would be interested in learning how to implement the extra support myself too, but I suspect this would be more time-consuming for you (I have no background in development), I'm afraid you would have to hold my hand all the way.

Side question: is havoc.cfg the best place to look to learn more about havoc features?

[ii8]

I'm stuck at home and have all the time in the world, so it's entirely up to you.

havoc.cfg just has a bunch of configuration options for appearance, what shell to use etc. It's commented so it should all make sense if you read it. havoc is minimal so there's not a lot of fancy features to talk about, it just does terminal emulation.

[Kabouik]

Then we can try a bit of both if you think that's relevant, maybe it wouldn't need to be spoonfed through every step if I can start by following your work, since I don't have much free time at the moment (but I can at least test new commits on the device in evenings).

Yes, havoc.cfg is pretty straightforward, that's convenient. What colour support does havoc feature? I'm also curious if it can handle ncurses' werase() function, because I am trying to package a CLI mail client that uses it and we noticed our current native terminal emulator (Fingerterm) refreshing gets wonky when werase() is used.

[ii8]

Ok, I made a branch called wl_shell only took 10 mins anyway, try it I guess, then we can fix any problems together.

havoc can do 24 bit colour and everything else you might want. werase should work, if not, make an issue for it and I'll look into it.

[Kabouik]

Awesome!

Here's what I get:

[nemo@Sailfish havoc-wl_shell]$ ./havoc 
could not open font file: No such file or directory
using fallback font
wl_registry@2: error 0: invalid version for global wl_seat (11): have 2, wanted 5
missing required ARGB8888 shm format

I hope it's not indicating other missing features of our compositor.

[ii8]

It is. wl_seat version 5 is what havoc uses, version 2 is what you have. It may just work if you change the version in the registry_get function where it calls wl_registry_bind for the seat. just change it to 2, see if it works.

Missing ARGB is a problem. That's required by the standard. Maybe it's there but the compositor is not reporting it, so you can just try deleting the test that produces the error.

[ii8]

Also, I pushed a small change to the wl_shell branch, I think I missed something earlier. So make sure you pull that.

[Kabouik]

Ok, so I did git pull, change the version to "2" on L257 of main.c, and deleted the two lines corresponding to the ARGB test, compiled again, and now ./havoc is throwing a segmentation fault (plus the minor missing font warning as it did before).

[ii8]

Huh, I guess that really means your compositor does not have ARGB8888 at all. The only way then is to use whatever other format it has and convert. There is a shm_format function, in there if you printf("%d\n", format); we'll see what formats there are.

[ii8]

Unless you added a segfault some other way, if you publish the diff, I can check.

[Kabouik]

I know nothing about it, but from the looks of it, I'd say that's not very encouraging:

[nemo@Sailfish havoc-wl_shell]$ ./havoc 
could not open font file: No such file or directory
using fallback font
0
1
Segmentation fault

[Kabouik]

[nemo@Sailfish havoc-wl_shell]$ git diff main.c
diff --git a/main.c b/main.c
index 2d40a88..eb0d3e5 100644
--- a/main.c
+++ b/main.c
@@ -1235,6 +1235,7 @@ static const struct wl_shell_surface_listener surf_listener = {

 static void shm_format(void *data, struct wl_shm *shm, uint32_t format)
 {
+printf("%d\n", format);
        if (format == WL_SHM_FORMAT_ARGB8888)
                term.shm_argb = true;
 }
@@ -1254,7 +1255,7 @@ static void registry_get(void *data, struct wl_registry *r, uint32_t id,
        } else if (strcmp(i, "wl_shell") == 0) {
                term.shell = wl_registry_bind(r, id, &wl_shell_interface, 1);
        } else if (strcmp(i, "wl_seat") == 0) {
-               term.seat = wl_registry_bind(r, id, &wl_seat_interface, 5);
+               term.seat = wl_registry_bind(r, id, &wl_seat_interface, 2);
                wl_seat_add_listener(term.seat, &seat_listener, NULL);
        } else if (strcmp(i, "gtk_primary_selection_device_manager") == 0) {
                term.ps_dm = wl_registry_bind(r, id,
@@ -1545,8 +1546,6 @@ retry:
                               " maybe you should use the xdg_wm_base havoc");

        wl_display_roundtrip(term.display);
-       if (term.shm_argb == false)
-               fail(eglobals, "missing required ARGB8888 shm format");

        cursor_init();

[ii8]

Something is wrong, but I don't know what. the 0 means ARGB8888, so not sure why it doesn't work. It looks like the contents of the shm_argb variable are lost.

Is it possible that the execution environment in this Sailfish OS doesn't allow static memory or something? That can't be it. Do you have some sort of debugger you can run? Or coredump?

Also, add the test for the ARGB format back in, so we can see if it happens before or after shm_format is called.

[Kabouik]

I still get the same output after restoring the ARGB check, and the ARGB warning doesn't show anymore so I guess it fails before:

[nemo@Sailfish havoc-wl_shell]$ ./havoc 
could not open font file: No such file or directory
using fallback font
0
1
Segmentation fault

I don't get the segmentation fault anymore (and the ARGB check is showing) if I restore wl_seat_interface to version 5, but then we're back to the beginning. We just know the version change causes the segmentation fault.

There probably are some debuggers available but I don't really know where to look. How do I check coredump on regular Linux? SFOS really is Linux (with some pretty outdated stuff as you can see).

[ii8]

https://wiki.archlinux.org/index.php/Core_dump I think sailfish uses systemd. They might be disabled.

[ii8]

I don't know if you can get gdb to run on sailfish OS, in that case it would be easier. Just run gdb with havoc and see where it crashes.

I'm in IRC rizon ##havoc, if you want to speed up the back and forth conversation.

[ii8]

@Kabouik check irc

[Kabouik]

@ii8 My Internet connection broke suddenly, and for some reason none of the phones in the house could get any Internet either. I'm sorry.

So after testing a little more, what I can tell from the current landscape hack under progress is:

• havoc does put SFOS in landscape mode when I launch it, or leaves it as it is if the OS already is in landscape mode; sounds good.
• iI has what seems to be a partly correct geometry for landscape: if I take the text orientation as a reference, then window width > height.
• However, the whole window (text orientation included) would need a CW 90° rotation to fit the actual screen orientation and ratio: right now when I hold my phone in landscape and launch havoc, the OS will stay in landscape but havoc is displayed as if I was holding the phone in portrait, and its width overextends beyond the screen height, while its height is not enough to fill the screen width.

Not related to this issue, but notes for later:

• I might have spoken too fast regarding the ncurses' werase() support; might have observed some issues in cmus despite no werase() in its code as far as I know. Will need to confirm when havoc is fully useable. [Edit] The issues with cmus are because havoc seems to enforce xterm-256color, which causes refreshing issues in SFOS. If I export TERM=xterm-color before running cmus, the refreshing issues are not here. Weirdly, this is already in my ~/.bash_profile, which I sourced again just in case, and the other SFOS terminal takes it into account but havoc (after make install) keeps using xterm-256color every time I restart it. werase() issues still here withxterm-color`.
• I think it ignores aliases set in ~/.bash_profile, may ignore all environment variables, but that is likely because I ran it from its build folder, before make install. [Edit] Confirmed solved after make install,
• Holding a key does not repeat it.

[Kabouik]

Also, @enigma9o7, a community member, pointed me to a hack he used to force landscape rotation on a game he built for SFOS. We should be able to avoid this for havoc since it's pure Wayland and shouldn't require qxcompositor, but here is what he used:

qxcompositor_pid()
        {
        # pgrep -u $HOST_USER -f 'qxcompositor --wayland-socket-name ../../display/wayland-1'
        pgrep -f 'qxcompositor --wayland-socket-name ../../display/wayland-1'
        }

{
qxcompositor --wayland-socket-name ../../display/wayland-1 &
export WAYLAND_DISPLAY=../../display/wayland-1
sleep 1
/usr/local/games/supertux2
kill "$(qxcompositor_pid)"
rm /run/user/100000/../../display/wayland-1.lock
}

Turns out if I change the binary to launch havoc instead of supertux2, then the terminal is correctly orientated. Its geometry is wrong, but we know we can swap width and height in main.c.

This is not ideal: it increases the startup time of havoc, it conflicts with Flatpak apps already running in SFOS (havoc will be superimposed), and it's qxcompositor, but maybe that can give some ideas.

[Kabouik]

This is after I swap height and width back in main.c and launch havoc with the qxcompositor script, it's using the full screen width and it's in landcape:

[ii8]

I pushed some changes to the sailfishos branch. It probably won't fix the landscape problem, but it's worth a try.

[Kabouik]

Thanks, I just tried it:

• that ones rotates the OS in portrait if it was previously in landscape, leaves it in portrait if it was in portrait;
• havoc shows up orientated in portrait;
• its window correctly fills the screen (correct geometry if it was meant to be showing in portrait).

When the window is shown in full screen (i.e., it has focus), in can confirm the OS considers itself in portrait. We have some OS-wide gestures to return to the multi-task view, show notifications, show some system toggles, or show the app launcher. I tested those gestures from inside havoc, and they are all correctly oriented for a portrait app. Note that this was the opposite before the changes you pushed in the last commit: the app would appear in portrait, but the OS would think it's in landscape, ultimately rotating the gestures too.

[ii8]

Yeah, that all makes sense. Looks like SailfishOS does not rotate apps like other mobile OSs do. Instead the client app has to do the rotation itself or draw everything on it's side directly.

[Kabouik]

Somehow the OS must have a way to rotate itself independently from apps though, because I can also rotate it when no app is maximized (multitask view, app launcher, events view), and that will rotate running apps accordingly. I can try to ask to SFOS developers later today.

[Kabouik]

For reference, this is the stock terminal for SFOS (well, not exactly the same version): https://git.sailfishos.org/mer-core/fingerterm/blob/master/main.cpp

And another one here: https://github.com/direc85/toeterm

This one should work and rotate on SFOS too but I need the SDK to build and try: https://github.com/rburchell/literm

They are all related in some ways.

[Kabouik]

There is something weird with the latest commit, check how cmus looks like compared to the first screenshot in landscape above:

Without tmux:

With tmux:

Colors and characters used for delimiters are different, even with tmux. Or is it just some scaling issue? Also the two screenshots show that there is a redraw issue with misaligned album durations when tmux is not used.

I don't want to pollute this issue since it's not related to building havoc on SFOS, but I'll open proper issues later if we can get a working SFOS build.

---

Scrap it. The issue must be somewhere else. My screenshots in portrait above showed that colour support was working correctly, now I get only basic colours if I run the same ls -la /usr/bin in havoc. With the same build. I must have broken something else on my device.

[ii8]

Those look like the default colors. It's probably unrelated to the latest commit and just not loading the right havoc.cfg.

[Kabouik]

I rebuilt and colours work again, I am sorry. I really don't know what happened, but I have been fiddling with ssh for hours and may just have broken something earlier (although I didn't touch to havoc binary).

The redraw issue when tmux is not used is confirmed though:

[Kabouik]

Alright, did some testing with it.

Without tmux

1. export TERM=xterm-color → Refreshing looks ok.

2. export TERM=xterm-256color → Refreshing broken. This is the default every time I run havoc from SSH, regardless of my ~/.bash_profile setting it to xterm-color. If I launch fingerterm from SSH, it's xterm-color.

3. export TERM=linux → Refreshing looks ok while inside cmus, but failure to hide the cmus interface when I quit it, I had to Ctrl+l to clear the view.

With tmux

export TERM=xterm-256color, export TERM=xterm-color and export TERM=linux all work the same: refreshing ok. I guess tmux is enforcing some other value that doesn't show when I issue echo $TERM.

[ii8]

Does havoc print any logs to stderr?

[Kabouik]

I edited my message above to add some details.

I just see this when navigating through cmus interface:

[nemo@Sailfish havoc]$ havoc
unknown DEC Set-Mode 1034
unhandled CSI sequence t
unknown DEC Set-Mode 1006
unknown DEC Set-Mode 1000
unhandled CSI sequence b
unhandled CSI sequence b
unhandled CSI sequence b
unhandled CSI sequence b

But that's printed in the SSH terminal from which I ran havoc, I'm looking how to check stderr logs.

[ii8]

That's it, that's what I wanted, ty.

[ii8]

ncurses was using the CSI b sequence to repeat characters, which is why the alignment was wrong. I implemented that in sailfishos branch now, force pull required.

[Kabouik]

I can confirm the refreshing issue is not occurring anymore when moving the selection of changing views in cmus.

Still some logs:

[nemo@Sailfish havoc]$ havoc   
could not open font file: No such file or directory
using fallback font
unknown DEC Set-Mode 1034
unhandled CSI sequence t
unknown DEC Set-Mode 1006
unknown DEC Set-Mode 1000
unknown DEC Reset-Mode 1006
unknown DEC Reset-Mode 1000
unhandled CSI sequence t
unhandled CSI sequence t
unknown DEC Reset-Mode 1000
unknown DEC Reset-Mode 1002
unknown DEC Reset-Mode 1003
unknown DEC Reset-Mode 1006
unknown DEC Reset-Mode 1005
unknown DEC Reset-Mode 1006
unknown DEC Reset-Mode 1000
unknown DEC Reset-Mode 1002
unknown DEC Reset-Mode 1003
unhandled CSI sequence q
unknown DEC Reset-Mode 1006
unknown DEC Reset-Mode 1000
unknown DEC Reset-Mode 1002
unknown DEC Reset-Mode 1003
unknown DEC Set-Mode 7727
unknown DEC Set-Mode 1006
unknown DEC Set-Mode 1000
unknown DEC Reset-Mode 1006
unknown DEC Reset-Mode 1000
unknown DEC Reset-Mode 1000
unknown DEC Reset-Mode 1002
unknown DEC Reset-Mode 1003
unknown DEC Reset-Mode 1006
unknown DEC Reset-Mode 1005
unknown DEC Reset-Mode 7727
unknown DEC Reset-Mode 1004
unhandled CSI sequence t
unhandled CSI sequence t
unknown DEC Set-Mode 1006
unknown DEC Set-Mode 1000

And as you can see my ~/.config/havoc/havoc.cfg with the correct font is being ignored right now, but I'm 97% sure it's human mistake on my side; will look into it. Right, it should be in ~/.config/havoc.cfg, my bad.

[ii8]

Don't worry about the other missing escape sequences. Terminals are a mess, there's hundreds of non-applicable and pointless things: https://www.xfree86.org/current/ctlseqs.html

[Kabouik]

Phew! So just missing landscape and repeat keys (repeat works in other terminals). I haven't received any answer on the #sailfishos Freenode channel so far regarding forcing landscape or enabling autorotation (probably not useful without a touch keyboard).

However, I believe the code you came up with does enforce an orientation regardless of the current OS orientation, which is what we want to have havoc always show in one direction only. It's currently enforcing portrait though.

In addition to the OS orientation, we would need havoc to rotate its content accordingly. So far, in all commits we tried, the text was always in portrait direction relative to the phone form factor (i.e., regardless of the OS orientation). The only thing that rotated havoc content in landscape relative to the phone form factor was the qxcompositor bash hack I posted above; not ideal.

[ii8]

I pushed a change to make pane resizing with arrowkeys work in tmux. But it was unrelated to key repeat, so that needs to be investigated separately.

nvm, it doesn't actually work.

[Kabouik]

I tried on phone before your edit, I confirm the behaviour is unchanged when pressing arrows while holding Ctrl (it just changes focused pane).

[Edit] Nevermind, I probably pulled after your removed the commit.

[ii8]

Nah, I didn't remove anything yet. The commit is probably still correct. I'm not sure, it implements how xterm behaves, but linux console and the vt in tmux behave different(the way havoc was before).

The resizing doesn't work with the linux console terminal either it seems.

[Kabouik]

Does this work on havoc master?

[ii8]

no, I added only to sailfishos branch so far

[ii8]

For some reason it works if I set $TERM=rxvt or $TERM=linux, but not with xterm or xterm-256color. Even though havoc matches the behavior of xterm, not the others.

[Kabouik]

I can confirm the same with the latest commit on device, but when I run havoc, it always ignores my ~/.bash_profile or prior manual export TERM=whatever and run with xterm-256color. Other terminals don't do that, is it somehow coded in havoc directly?

Any disadvantages in using rxvt or linux despite havoc matching the behavior of xtem only?

[ii8]

Other terminals don't do that, is it somehow coded in havoc directly?

They should, whenever you run a terminal it should set the TERM variable. If you run tmux for example and then echo $TERM inside, it should say screen. So exporting TERM before won't work because it will be overridden, you have to set it in the shell after running havoc. I don't know about bash, you shouldn't really set TERM in there, because you don't know what terminal bash is running in so it could be wrong.

Any disadvantages in using rxvt or linux despite havoc matching the behavior of xterm only?

Probably, there will be incompatibilities in other places. Terminals are all slightly different, it gets very confusing.