Add macOS workflow

[valentinegb]

Adds a workflow which builds and uploads a macOS artifact of Tuba on push or pull request.

[GeopJr]

Thanks!

I have a few questions:

• Is there any way to run the tests? If it's troublesome, it's okay to skip them!
• Is the way it's packaged 'good enough'? I'm guessing, but since the zip doesn't include the adwaita icon theme then icons are going to be missing(?). FWIW, this is what we do for windows https://github.com/GeopJr/Tuba/blob/main/Makefile#L63-L75 (copying all icons and then attempting to clean up anything we don't need)

[valentinegb]

Is there any way to run the tests? If it's troublesome, it's okay to skip them!

Pretty sure I can do that, I will next

Is the way it's packaged 'good enough'?

Now that I think about it more, not really ;-;

A Mac user who downloads an artifact from this workflow would, on their machine, needs to make sure they have all the runtime dependencies installed, link the share directory (I don't think you technically need to link bin), and run the post-install commands which, I've just realized, for Mac are only explicitly showed in the Homebrew formula, so it'd be a good idea probably to add something like an install script to do all that for the user and include that in the artifact I can start working on that next

since the zip doesn't include the adwaita icon theme then icons are going to be missing(?)

If the user has the dependency installed, adwaita-icon-theme, then they should appear fine in the app

this is what we do for windows https://github.com/GeopJr/Tuba/blob/main/Makefile#L63-L75 (copying all icons and then attempting to clean up anything we don't need)

I think this could be done for Mac as well, but should we if we're going to install all of the app's runtime dependencies for users?

[valentinegb]

Okay instead of making an install script, I'm going to focus on getting the minimum working app bundle with dependencies included in the bundle, that way user installation is super easy

[valentinegb]

There should be some application code to set the XDG_DATA_DIRS environment variable to ../Contents/Resources/share if running on macOS so that it searches for files like settings schemas and icons inside the app bundle and not in /usr/share like is default (I think) I would really appreciate it if someone else did that since I have never touched Vala before and my editor doesn't even have syntax highlighting for it yet 😅

As of right now though, if you download the app bundle from the macOS workflow on a Mac and set XDG_DATA_DIRS manually + have runtime dependencies (you might also have to run chmod +x Tuba.app/Contents/MacOS/dev.geopjr.Tuba, actions/upload-artifact strips permissions :P), it should work! Next is bundling runtime dependencies, too.

[GeopJr]

Bundling the dependencies is very messy... See on windows https://github.com/GeopJr/Tuba/blob/main/Makefile#L46-L57, we go through every single linked dep and straight up copy them and their dependencies to the folder

There should be some application code to set the XDG_DATA_DIRS environment variable to ../Contents/Resources/share if running on macOS so that it searches for files like settings schemas and icons inside the app bundle and not in /usr/share like is default (I think)

I believe, like on windows, it will search locally first, as long as the folders are structured right (see the windows zip package!). Doesn't it do it already for Tuba's own icons since they are included in the artifact? I'll investigate

strips permissions

( :shushing_face: it keeps them if you zip it yourself)

---

Honestly, I don't know if it's worth it to bundle everything. Actually, I don't even know if it's worth it to ship it outside of homebrew at all (except for testing)

Windows is a different beast, all dependencies have to be provided from msys2 and you can't really expect users to go through all that. But I suspect that most mac users willing to download an app outside of the appstore, also have homebrew installed :shrug:

I'm setting up a Mac KVM and I'll see tmrow!

GNOMies seem to be using https://gitlab.gnome.org/GNOME/gtk-mac-bundler to bundle mac apps, might be worth considering. Actually, taking a look on how gimp and inkscape do it might be helpful too!

[valentinegb]

Bundling the dependencies is very messy...

Yeah I've realized that now lol Still, I think I'm making progress on it, so I think I'll keep trying. I learned that macOS app bundles have a Frameworks directory where dynamic libraries are intended to go, so now I'm utilizing that (not pushed yet)

I believe, like on windows, it will search locally first, as long as the folders are structured right

How exactly should they be structured then? Right now it's like:

Tuba.app
    Contents
        MacOS
            dev.geopjr.Tuba
        Resources
            share
                ...
                icons
                ...

Doesn't it do it already for Tuba's own icons since they are included in the artifact? I'll investigate

Not in my testing. But yeah, feel free to download those artifacts and test stuff! ^^

it keeps them if you zip it yourself

Oh nice okay I'll have to add that then

Honestly, I don't know if it's worth it to bundle everything

It would be really nice, and maybe not so far off actually since I've made a ton of progress today

But I suspect that most mac users willing to download an app outside of the appstore, also have homebrew installed 🤷

The thing with that, though, is that Homebrew formulas (compiles a program from source) can't really have app bundles and they're not even actually meant for GUI applications, only CLI and libraries (hence why the current Tuba formula only allows users to open Tuba through Terminal). A Homebrew cask (downloads a release artifact) is really what I'm aiming for, which is intended specifically for GUI applications.

GNOMies seem to be using https://gitlab.gnome.org/GNOME/gtk-mac-bundler to bundle mac apps, might be worth considering. Actually, taking a look on how gimp and inkscape do it might be helpful too!

I have looked at gtk-mac-bundler and GIMP! I learned some stuff from looking at how gtk-mac-bundler works, but I continued implementing things myself, probably mostly because I kinda hate how GIMP is on Mac- it's all kinds of janky

[GeopJr]

How exactly should they be structured then?

Not in my testing

My bad, I thought we were still following the original structure (/whatever meson outputted), where the dependencies sit next to the binary and the others on ../share/ ...

The thing with that, though, is that Homebrew formulas (compiles a program from source) can't really have app bundles and they're not even actually meant for GUI applications, only CLI and libraries (hence why the current Tuba formula only allows users to open Tuba through Terminal). A Homebrew cask (downloads a release artifact) is really what I'm aiming for, which is intended specifically for GUI applications.

I see, thanks for the info!

---

Btw, I'll start pushing to this PR if I have something to push, so don't really worry about the commit history, it will be squashed anyway!

[GeopJr]

Okay so, I did more research and what I'd like would be to follow the windows pattern of using a Makefile (can be done after we figure out all the commands):

mac: PREFIX = $(PWD)/tuba_mac/Tuba.app/Contents/Resources
mac: __mac_pre build install # ...

__mac_pre:
    rm -rf $(PREFIX)
    mkdir -p $(PREFIX)/lib/
    mkdir -p $(PREFIX)/../MacOS/
    touch $(PREFIX)/../Info.plist # cp build_aux/...

build_aux can hold mac specific scripts/files/icons

And what we should follow is Kangaroo's guide: https://www.datatable.online/en/blog/004-how-to-deploy-gtk-app-on-mac.html

Kangaroo is a commercial GTK app built using (almost) the same stack as Tuba - Vala, gtk, gtksourceview... So that guide will be on point!

Oh btw, could you add libspelling to the deps? (spelling library, not sure if it will work, but just add it anyway). And maybe webp-pixbuf-loader for webps

---

So the plan is:

1. Get it into a working state (incl. icons, gtksourceview langs and themes, webp loader, videos)
2. Cleanup
3. Use big sur icon
4. Create DMG

[valentinegb]

Any idea why brew --prefix adwaita-icon-theme seems to do nothing in Makefile? .-.

[GeopJr]

You need to prefix it with another $ so it becomes $$(brew ...)

(one $ is for substitution of its own variables (that's why $(PREFIX) works), while $$ will escape it)

[valentinegb]

You need to prefix it with another $ so it becomes $$(brew ...)

Ohh thanks! This is my first time writing a Makefile 😅

[valentinegb]

We should be at the point now where the app bundle is completely standalone and doesn't require the user to install any dependencies! A little bit hard to test on my Mac though since I have other stuff that still needs a lot of the same dependencies, so please test with an environment lacking the required dependencies if you can!

[GeopJr]

Thanks for all your work on it! :bow:

A little bit hard to test on my Mac though since I have other stuff that still needs a lot of the same dependencies, so please test with an environment lacking the required dependencies if you can!

I'm on it!

[GeopJr]

Won't start, this is all I got from console:

dev.geopjr.Tuba-2024-05-28-192436.ips.txt

I don't know how to read it but my guess is that the paths aren't updated and instead point at your brew installation(?) "path" : "\/usr\/local\/Cellar\/cairo\/1.18.0\/lib\/libcairo-gobject.2.dylib"

[valentinegb]

Ruh roh- try running ./Tuba.app/Contents/MacOS/macos_wrapper.sh, that should give much more useful output

[valentinegb]

I don't know how to read it but my guess is that the paths aren't updated and instead point at your brew installation(?)

Yeah app bundle error logs are kind of useless honestly lol but I don't think it's missing a dynamic library, I know what the error for that looks like in those logs and if it were that I would probably be at a loss as to how it could be failing to find any dependencies now It's probably more of a runtime error, at the point where GTK is giving the error. Maybe there's a certain share directory that needs to be included?

[GeopJr]

I think I found the culprit, CI builds are arm64, my qemu vm is x86-64

Could you send me a build from your machine, if it's x86-64? Else I'll re-install brew, build it and remove it again

edit:

running that script gave me bad CPU type in executable, running file ... returned arm64

[valentinegb]

Ah, yep that would do it My machine is also arm64 but what I could do is modify the workflow to build both x86-64 and arm64, upload those artifacts, and then I could also have another job that downloads both artifacts, combines them into an executable that works on both architectures (but is twice the size) and upload that as a third artifact (pretty commonly done on Mac, called a universal binary)

[GeopJr]

Sounds good! I'm okay with either separate builds or a universal one!

[valentinegb]

First x86-64 artifact just got uploaded, try that when you can ^^

[GeopJr]

Worked!

Issues:

• Icons don't load, according to the following logs, the pixbufloader is missing
• TLS is not available, I believe glib-networking is responsible for that

:thinking: Both of these should be included if the script recursively bundled deps :/ not sure why it doesn't find them

[valentinegb]

🤔 Both of these should be included if the script recursively bundled deps :/ not sure why it doesn't find them

I think the problem is that the library files it's mentioning in the console are normally in subdirectories within lib, and when dependencies are recursively copied it either doesn't copy things in further directories or it does but puts them straight into Resources/lib, without their subdirectories being replicated too

[valentinegb]

Oh, actually, I don't even see libpixbifloader-svg.so installed on my machine at all .-.

[GeopJr]

://///// try installing librsvg if not already installed

but actually, if they (svgs) work when you run it then I really don't get it

[valentinegb]

Ah, I have librsvg installed, by coincidence, so that's why it worked on my machine I'll add it to Brewfile and see how things improve

[valentinegb]

Currently rewriting the dependency copying stuff to just copy everything in $HOMEBREW_DEPENDENCIES, since brew bundle creates an environment that only has access to packages listed in Brewfile, so it shouldn't really copy any unnecessary libraries I think

[GeopJr]

Update:

• TLS issue is fixed! I was able to login
• Icons are still broken (same log as before)
• Unpacking it results in 800+ MB :(
• Segfaults right when it's about to finish loading posts, might be pixbuf or qemu related(?), setting up gdb was hellish last time I tried so I don't think I'll thoroughly debug this (last time I had to use self signed certificates and similar)

[valentinegb]

Icons are still broken (same log as before)

So, for some reason, it looks in subdirectories when looking in /opt/homebrew/lib but it doesn't in $(PREFIX)/lib, it just looks in that directory, so I guess we gotten figure out how to flatten it or something

Unpacking it results in 800+ MB :(

Yeeaaaaah ;-; that's why I said "experiment" in the commit lol. Not really sure if it's viable, but I know that at least a portion of that is actually build dependencies, which don't need to be included. We could probably have two Brewfiles, one with both build and runtime dependencies and one with only runtime dependencies

Segfaults right when it's about to finish loading posts

Oh odd, I thought I had removed most dependencies from my machine and it works mostly fine for me (I am experiencing the icon issue now)

[valentinegb]

Hah, so, TLS just broke for me for the first time Did you do something to fix it for yourself or did it just fix itself in 0f8bbfc?

[GeopJr]

The kangaroo guide is probably our best bet at a minimal and complete bundle but between us only you have a working Mac so the implementation is up to you :sweat_smile:

did it just fix itself in https://github.com/GeopJr/Tuba/commit/0f8bbfc885172fa3e28bd2b70a5e955eaec47e5e?

Yep, make sure glib-networking was bundled / exists

[valentinegb]

You know what, I hate to go back on what I said before but trying to get every little thing that the app could possibly need to run correctly bundled in with it is such a pain, takes up way too much space, and Homebrew casks do allow you to specify dependencies (which I kind of forgot about...) And you're probably right that Mac users that want to use Tuba will probably have Homebrew already anyway, considering Mastodon users are generally more tech savvy I think In any case, I'm gonna go and say that it's not worth all the effort to keep trying to make the app bundle completely standalone. Sorry for kind of wasting time ;-; I'll remove all the dependency copying stuff and everything should work perfectly if you run brew bundle in the root of the repository (my fork) and open the app

[valentinegb]

Just included the Brewfile in distributions, so now you can run brew bundle in the distribution DMG and not download the whole repository

[GeopJr]

It's ok! Wish I could be of more help :(

I'll give it a try myself too if I get some free time, but for the moment it's good enough! It's not like we can't come back to it in the future anyway

Thank you for taking over this task :bow:

[GeopJr]

I gave it another try, this is the closest I got to replicating the windows build

It copies over all needed deps, icons, loaders and then cleans up. I did encounter some issues though with the dep loops. Specifically with some icu data stuff and a libwebp dep, maybe we can make otool ignore them if they are not needed? or copy them manually instead

If we can make it work in this state, then we can move on to creating Tuba.app and the installer

(oh I also installed libproxy, not sure if it's needed tbh)

macos: PREFIX = $(PWD)/tuba_macos
macos: MAC_APP = $(PWD)/Tuba.app
macos: __macos_pre build install __macos_copy_deps __macos_schemas __macos_copy_icons __macos_cleanup

__macos_pre:
    rm -rf $(PREFIX)
    mkdir -p $(PREFIX)/lib/

__macos_prepare_app:
    # mkdir -p $(MAC_APP)/Contents/
    # mv $(PREFIX) $(MAC_APP)/Contents/Resources/

    # mkdir -p $(MAC_APP)/Contents/Resources/lib/
    # mkdir -p $(MAC_APP)/Contents/Resources/bin/
    # mkdir -p $(MAC_APP)/Contents/Resources/share/
    # mkdir -p $(MAC_APP)/Contents/Resources/etc/
    # mkdir -p $(MAC_APP)/Contents/Resources/include/
    # mkdir -p $(MAC_APP)/Contents/MacOS/

__macos_copy_deps:
    otool -L $(PREFIX)/bin/dev.geopjr.Tuba | grep "/*.*dylib" -o | grep -v "/usr/lib/*" -o | grep -v "/System/Library/*" -o | xargs -I{} cp -nL "{}" $(PREFIX)/bin

    cp -fL $${HOMEBREW_PREFIX}/opt/webp/lib/libwebp-7.dylib $(PREFIX)/bin
    cp -fL $${HOMEBREW_PREFIX}/opt/librsvg/lib/librsvg-2.2.dylib $(PREFIX)/bin
    cp -fL $${HOMEBREW_PREFIX}/opt/gmp/lib/libgmp.dylib $(PREFIX)/bin
    cp -fL $${HOMEBREW_PREFIX}/opt/gnutls/lib/libgnutls.dylib $(PREFIX)/bin
    cp -fL $${HOMEBREW_PREFIX}/opt/glib/lib/libgthread-*.dylib $(PREFIX)/bin
    cp -fL $${HOMEBREW_PREFIX}/opt/libproxy/lib/libproxy.dylib $(PREFIX)/bin
    cp -frL $${HOMEBREW_PREFIX}/opt/gdk-pixbuf/lib/gdk-pixbuf-2.0 $(PREFIX)/lib/gdk-pixbuf-2.0
    cp -frL $${HOMEBREW_PREFIX}/opt/webp-pixbuf-loader/lib/gdk-pixbuf-2.0 $(PREFIX)/lib/gdk-pixbuf-2.0

    # Let's recursively get all deps of deps 3 levels deep
    for dep in $$(find $(PREFIX)/bin -name \*.dylib -o -name \*.so -type f); do \
        otool -L $$dep | grep "/*.*dylib" -o | grep -v "/usr/lib/*" -o | grep -v "/System/Library/*" -o | xargs -I{} cp -nL "{}" $(PREFIX)/bin; \
    done

    for dep in $$(find $(PREFIX)/bin -name \*.dylib -o -name \*.so -type f); do \
        otool -L $$dep | grep "/*.*dylib" -o | grep -v "/usr/lib/*" -o | grep -v "/System/Library/*" -o | xargs -I{} cp -nL "{}" $(PREFIX)/bin; \
    done

    for dep in $$(find $(PREFIX)/bin -name \*.dylib -o -name \*.so -type f); do \
        otool -L $$dep | grep "/*.*dylib" -o | grep -v "/usr/lib/*" -o | grep -v "/System/Library/*" -o | xargs -I{} cp -nL "{}" $(PREFIX)/bin; \
    done

    cp -f $${HOMEBREW_PREFIX}/opt/gtksourceview5/share/gtksourceview-5/styles/Adwaita.xml $${HOMEBREW_PREFIX}/opt/gtksourceview5/share/gtksourceview-5/styles/Adwaita-dark.xml ${PREFIX}/share/gtksourceview-5/styles/
    cp -f $${HOMEBREW_PREFIX}/opt/gtksourceview5/share/gtksourceview-5/language-specs/xml.lang $${HOMEBREW_PREFIX}/opt/gtksourceview5/share/gtksourceview-5/language-specs/markdown.lang $${HOMEBREW_PREFIX}/opt/gtksourceview5/share/gtksourceview-5/language-specs/html.lang ${PREFIX}/share/gtksourceview-5/language-specs/

__macos_schemas:
    cp -r $${HOMEBREW_PREFIX}/opt/gsettings-desktop-schemas/share/glib-2.0/schemas/*.xml ${PREFIX}/share/glib-2.0/schemas/
    glib-compile-schemas ${PREFIX}/share/glib-2.0/schemas/

__macos_copy_icons:
    cp -r $${HOMEBREW_PREFIX}/opt/adwaita-icon-theme/share/icons/ $(PREFIX)/share/icons/
    cp -r $${HOMEBREW_PREFIX}/opt/hicolor-icon-theme/share/icons/ $(PREFIX)/share/icons/

__macos_cleanup:
    rm -f ${PREFIX}/share/glib-2.0/schemas/*.xml
    rm -rf ${PREFIX}/share/icons/hicolor/scalable/actions/
    find $(PREFIX)/share/icons/ -name *.*.*.svg -not -name *geopjr* -delete
    find $(PREFIX)/lib/gdk-pixbuf-2.0/2.10.0/loaders -name *.a -not -name *geopjr* -delete
    find $(PREFIX)/share/icons/ -name mimetypes -type d  -exec rm -r {} + -depth
    find $(PREFIX)/share/icons/hicolor/ -path */apps/*.png -not -name *geopjr* -delete
    find $(PREFIX) -type d -empty -delete
    gtk4-update-icon-cache $(PREFIX)/share/icons/Adwaita/
    gtk4-update-icon-cache $(PREFIX)/share/icons/hicolor/