MacOS Certificate Issue

[ABaumher]

Currently, MacOS gives a security error for our cpython version. It usually looks like:

It is possible to adjust security settings to trust our app, but it would be preferable to not have this issue.

[ABaumher]

@bgebhardt

There are three fixes i can think of - including certifi in the final product, using python 3.7.9 installed from https://www.python.org/downloads/release/python-379/ instead of brew, or running the install certificate commands. I have no idea which would be the most effective.

[Mr-Spock1]

Afaik you can use brew with the --noquarantaine option to bypass it. Haven't testet it though.

[ABaumher]

Iirc (second-hand, could be wrong) you can't brew for python 3.7 since it's in long term support and the newer versions are preferred. But you can get pyenv and install it through there. Afaik pyenv doesn't have that quarantine option so we might be out of luck there. I'm not sure if the pkg version on python's site is signed and would work without issue. In theory certifi is just a collection of certificates and whatever python needs to do to make apple or Linux recognize them. So maybe if we pack with that as part of our apple release it'll stop complaining, but i really don't know without testing it

[ABaumher]

I hope to god this version works. I've got a Mac I can build on (occasionally) and afaik did not agree to any certifications. It works here. I built it using certifi and python 3.7.9 directly from the python downloads. Maybe just certifi is enough but idk. I need someone else to give it a try because it might not be a security on this Mac because I did the development here.

steam_v1.0.5.zip

An aside, we definitely need to update pip-tools to use a newer version because it will not work on MacOS at the current version. Credit due to #10 and #11 (I'd get the name of the author but I am extremely uncomfortable on a Mac and Safari is giving me enough trouble as it is).

[Mr-Spock1]

Replaced my working 1.0.4 steam plugin folder with the new one (renamed it accordingly of course), got 14 popups and the plugin crashed.

[ABaumher]

Well that sucks. It worked on the Mac i tested with but who knows why, lol. I'm guessing every popup was security related?

[ABaumher]

Per https://developer.apple.com/developer-id/ it appears i need an apple developer account in order to codesign the code so it's allowed. Which runs a cool $99 per year. Yeah... I'm not doing that. In theory python is all runtime code, nothing needs to be compiled and therefore nothing is quarantined. But that appears not to be the case.

@bgebhardt it appears i was right - since you compiled the plugin yourself the "quarantine" flag that apple enforces on all unsigned apps is cleared.

I'm curious - if we unzip the plugin and take the .so files from the old code, would it work? Technically the python code changed so we should break the notarization check, but again i don't think py files are compiled and don't need to be signed.

[Mr-Spock1]

I'm pretty sure if I go through the whole permission granting process with lots of Galaxy restarts like I did with 1.0.4 the plugin will eventually run fine. I'll try that in a few hours and report back if that's helpful.

[ABaumher]

The goal was to skip all the allow dialogs but it appears my code approach has the same results as the 1.0.4 and previous 1.0.5 so i didn't do anything lol.

I've thought up a few possibilities: if you delete the plugin file so GOG reinstalls the old broken plugin and just modify the existing files, does it still go nuts about security? Because if not, the only thing gatekeeper cares about is the .so files and afaik they haven't changed. I could just steal those from the old build and pack them into the zip instead.

Alternatively, since i didn't try the install certificate commands (i hoped including certifi would do that) i could try that before packing. Or i could pack them into the zip and force Mac users to run them as part of the install instructions. Far from ideal but better than me paying $100 a year to maintain this project

This means I'll need to borrow a Mac again some point later this week. What a joy

[sparkus71]

I'm curious - if we unzip the plugin and take the .so files from the old code, would it work? Technically the python code changed so we should break the notarization check, but again i don't think py files are compiled and don't need to be signed.

FWIW, I downloaded the latest mac build, deleted every .so file, and copied everything else over the original plugin, and the plugin crashes on start.

I then found this: https://developer.apple.com/forums/thread/713051 and ran this command find . -name "*.so" -exec xattr -c {} \; on the plugin to clear the quarantine attribute on the .so files and now it works.

[ABaumher]

Huh. I wasn't really planning it but i could add a shell script that makes the install a one-step solution. It'd automatically unzip the file, copy the contents to the right directory, and remove any old stuff that's not used any longer. If i can run that find command directly in a shell i could do it there as well. I'm guessing the shell script itself would fail gatekeeper but imo executing mystery shell scripts should raise a security issue. One allow security prompt in exchange for a fully automated install seems like a fair trade

[spacewaves]

I'm curious - if we unzip the plugin and take the .so files from the old code, would it work? Technically the python code changed so we should break the notarization check, but again i don't think py files are compiled and don't need to be signed.

FWIW, I downloaded the latest mac build, deleted every .so file, and copied everything else over the original plugin, and the plugin crashes on start.

I then found this: https://developer.apple.com/forums/thread/713051 and ran this command find . -name "*.so" -exec xattr -c {} \; on the plugin to clear the quarantine attribute on the .so files and now it works.

Hey! It works great. Thank you. @ABaumher also thanks for your build ;)

[Mr-Spock1]

I'll postpone the installation of 1.0.5 until the shell-solution is ready. Otherwise I can't test if the script works or not when I already solved every permission beforehand.

[ABaumher]

I'm going to need to check with the humble bundle plugin dev, but I'm curious if installing it through gog galaxy would avoid the security issue. Getting it set up so you can search github through gog for my version (replacing the default one seems like a pipe dream atm) is the next step towards stable, as it's possible to auto update once gog hooks into the repo. I'm not exactly sure how yet but I'll look into it.

If gog is certified (which it is) the stuff it installs might not be blocked. I'm guessing that's not the case because apple likes being an asshole wants to keep you safe at the cost of making an accessible platform for development, but it might be. If it still blocks and the script hack doesn't work, maybe one of the other plugin devs has an active apple developer account and i could convince them to compile this plugin for me.

I'm definitely going to try the shell script approach first, regardless, but gog's autoupdate would like reapply the quarantine flag so it's something I'll have to address at some point.

[ABaumher]

From what i hear from the Humble Plugin dev, the GIthub Actions auto-build tool may automagically fix this. I have no idea how. In the meantime, i've written a script that maybe works?

EDIT: Updated this file at 6:16 PM UTC (UTC+0) Sunday, June 4, 2023 testme.zip

@Mr-Spock1 can you test this for me? If it works i'll zip these together but for now, i just want you to try it out manually.

1. Retrieve the latest mac version from https://github.com/ABaumher/galaxy-integration-steam/releases/tag/v1.0.5-beta
2. rename it to mac.zip if it is not already called that.
3. Retrieve the shell script from that zipped file above. You will need to unzip it first.
4. Put the shell script and the zipped beta release in the same directory. it does not matter where.
5. Run the shell script.

I'd actually recommend opening the shell script at step 5 instead of running it, and then running each command manually, to make sure i didn't make a mistake in the first command that will somehow break the rest. It looks proper on my WSL Ubuntu instance, but Linux and MacOS (Unix) are not perfectly identical. It's a shell script instead of bash like i'm used to but iirc MacOS switched to zsh instead of bash so i need to cater to both. You can also prepend "echo" to each command to see what it's trying to do without running it.

[ABaumher]

...And windows doesn't have a nice "unzip" call. what a joy. Time for me to test the ported script on Windows.

Edit: but powershell does. I have a windows script that works. Lmk on the macos script and i'll push both to the repo when they both work

[Mr-Spock1]

I tried testme.sh but it didn't work for me (both line by line and as a whole). I then had a conversation with ChatGPT where, after I told it what the terminal was spitting out, it suggested the following customization:

#!/bin/sh

SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd)"
MAC_ZIP="$SCRIPT_DIR/mac.zip"
PLUGIN_PATH="$HOME/Library/Application Support/GOG.com/Galaxy/plugins/installed/steam_ca27391f-2675-49b1-92c0-896d43afa4f8"

mkdir -p "$PLUGIN_PATH"
unzip "$MAC_ZIP" -d "$PLUGIN_PATH"
find "$PLUGIN_PATH" -name "*.so" -exec xattr -c {} \;

And lo and behold, that worked on the first try! I forgot: I had to set the execution flag on testme.sh after unzipping it. Don't know if it was set and cleared by unzip. And the plugin is reported as 1.0.4.0 by Galaxy.

[Mr-Spock1]

Oh, ChatGPT forgot the deletion part. But I think you can restore that. I just noticed when trying a second time that I have to choose if I want to replace existing files.

[Mr-Spock1]

What might have caused my first fail: My downloads go to a RAM disk by default to minimise unnecessary SSD writes.

[Mr-Spock1]

For completeness and with apologies from the AI:

#!/bin/sh

SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd)"
MAC_ZIP="$SCRIPT_DIR/mac.zip"
PLUGIN_PATH="$HOME/Library/Application Support/GOG.com/Galaxy/plugins/installed/steam_ca27391f-2675-49b1-92c0-896d43afa4f8"

rm -rf "$PLUGIN_PATH"
mkdir -p "$PLUGIN_PATH"
unzip "$MAC_ZIP" -d "$PLUGIN_PATH"
find "$PLUGIN_PATH" -name "*.so" -exec xattr -c {} \;

ROFL

[ABaumher]

Not sure how i feel about "AI" (that term is a misnomer) coding for me, but it makes sense. i had to do something similar with the windows version with $~dp0 or whatever to get the script's path. If Mac's shell execution does not start in the script directory you need to point it there. Windows defaults to either the user folder or system32, i assume Mac does something similar.

I don't actually know if chmod carries over from a browser download. Regardless, i zipped it on windows so that may not have helped. I'll need to do that in WSL Ubuntu if i really want to make sure

Edit: I can use tar if that's preferred on MacOS. It's possible to keep chmod on a zip but i'll need to figure that out.

[ABaumher]

The plugin being reported as 1.0.4 is a known error on my part, i need to keep a config file up to date and i missed the mac version. i think this was because at the time i did not have a mac version 1.0.5 (i don't have the ability to compile them myself most of the time), so i just put it off. 1.0.4 > 1.0.3 so GOG won't try overwriting it, and therefore it's only a display issue.

[Mr-Spock1]

Some feedback to the Mac instructions: The downloaded file auto-copy-mac.sh has no x-flag and Finder can only set read/write permission. Luckily you don't need admin rights and no sudo to use chmod. Just open Terminal, write chmod +x (with the blank) and drag auto-copy-mac.sh in the Terminal and press Enter. Then drag it in again and run it. Maybe my attached zip keeps x-flag… have to test that: auto-copy-mac.sh.zip …seems to work here. Unfortunately the script will be opened with Xcode by default when double clicked. I don't know which programm will open it when Xcode is not installed. So you have to open Terminal and drag the script in.

[ABaumher]

Does command+click give you the option to run the script? I still don't understand why Apple won't use right click instead of command click but w/e

Rename the shell script by removing the .sh. It should run by default when you double click it now

[ABaumher]

I didn't just wrap the sh script and zipped plugin in a zip file because i was afraid the unzip process would remove the execute permission but maybe i was wrong there. I know you need to specify a flag (i believe -X) when using unzip to keep permissions. Idk if finder does that by default

[Mr-Spock1]

Command click is for making a selection. It's control click which essentially is the right click on mac. There is indeed a open with… menu but the Terminal is not among the presented apps. You can choose other and then navigate to Utilities and once in, when you see Terminal you can't select it because you have to switch from "recommended" to "all Apps" first.

[Mr-Spock1]

Rename the shell script by removing the .sh. It should run by default when you double click it now

That's right but deleting the ".sh" via Finder rename just hides the extension. You have to open the Info window and delete the extension there. And you have to confirm that you really want to remove it. Then the icon changes to a small black Terminal window and you can double click it.

[Mr-Spock1]

Idk if finder does that by default

When I download my zip and double click to extract it kept the x-flag. Let me test this with the no-extension-script… auto-copy-mac.zip … the x-flag is still set and it executes by default BUT the script is now quarantained bummer Edit: my first zip leads to a quarantained version es well. It worked because I dragged it in the Terminal I believe. Edit2: Confirmed. Dragging works while double click leads to the "unknown developer" error

[Mr-Spock1]

The current extensionless auto-copy-mac has again no executable flag and it now opens with TextEdit by default. I think the best compromise so far is a script with .sh and x-flag zipped and the user has to double click to extract, open Terminal and drag the script in it's window and press Enter.

[ABaumher]

Another option i saw was adding .command to the shell script (changing the file extension if finder doesn't do it by default) but i have no idea if that'd work. proccing gatekeeper would be OK if Apple gives you the option to just say "it's ok, allow it" because you're running a mystery script, it should make sure you trust it first.

From the documentation I've seen extensionless should work assuming it's got executable permission. Unfortunately i can't chmod on Android with termux and make it stick outside of termux's sandbox, so any upload i do is through windows, which likes to throw out chmod permissions. Unless it's zipped. I probably could do that through wsl or use tar which Microsoft doesn't mess with

[ABaumher]

My workaround for this issue is going to be making a tar for MacOS that includes the .command and mac.zip. Tar will keep the permission on MacOS, then the command unzips everything to the new directory and clears the quarantine flags. The command will proc gatekeeper, but if you can just allow that with a click, that seems ok. It's roughly equivalent to a UAC prompt in Windows.

[Ostainia]

Idk if finder does that by default

When I download my zip and double click to extract it kept the x-flag. Let me test this with the no-extension-script… auto-copy-mac.zip … the x-flag is still set and it executes by default BUT the script is now quarantined bummer Edit: my first zip leads to a quarantined version as well. It worked because I dragged it in the Terminal I believe. Edit2: Confirmed. Dragging works while double click leads to the "unknown developer" error

Dude the zip worked perfectly for me! I couldn't get the .command script to give proper access so I started poking around to see if I could find a workaround. The unix executable will run if you verify it by right clicking to open and after that there's no problems at all. Tysm :)

[ABaumher]

@urwrstkn8mare since you are on a Mac, you might have more knowledge on this than i do.

This is largely why that auto-copy-mac.command exists in the repo. I haven't been able to find a better solution. It's possible this affects Humble as well, idk. UncleGoogle said it didn't affect him as far as he knew but i think his stuff was from before Calatina decided to be such a pain in the a**

[urwrstkn8mare]

yeah it didn't affect me either it just worked except it still doesn't show installed games