search

adventuregamestudio / ags

AGS editor and engine source code
Other
695 stars 159 forks source link

[Q] Will it make sense to supply a macos engine with the release? #1333

Open ivan-mogilko opened 3 years ago

ivan-mogilko commented 3 years ago

This was bothering me for a while, I know that CI is building mac port binary, but we don't have it uploaded to the github releases, neither posted in release thread on forums etc. That probably makes it less likely for users to know of its existence.

I'm not a mac user, but I heard that normally a mac app should be packaged in a certain way to become a legitimate osx app.

But still will it work if we provide a raw engine binary for things like quick introduction, random game testing, and so on?

Which steps exactly are required to go from a raw binary to a usable program on mac?

UPDATE:

After speaking with few people on Discord I realized that one has to make a proper mac app "wrapper", as also explained in the readme.

So my question is updated to: will it make sense to provide such "wrapper" with the release for easier testing of your game in mac? Even if this app is not signed, it still may run, seems like user only has to give system an order to force launch it on purpose.

ericoporto commented 3 years ago

the steps that are done like mygame.app and the info.plist could be generated by the editor, but the codesigning and the notarization which ARE needed to release for macOS afaik requires a MacOS computer to do the steps.

On packaging, the best information is here : http://www.edenwaith.com/blog/index.php?p=112 , so up to until the paragraph that reads Code Signing I believe is possible to do in the Editor.

hobbesjaap commented 3 years ago

It's my understanding that if I create the app, I can run it on my Mac without notarization & codesigning. To release wider, these two things do need to happen. Unless we want people to have to run xattr -cr commands on unsigned apps every time they download them.

So for an unsigned build, this would work I think. I've currently got the "compile for Mac" thing done as posted on the forum the other day. I've now symbolic-linked my "Data" folder (when I compile my game in the editor) to the "Resources" folder in the Mac game wrapper. So whatever I do, I get a ready-to-go Mac-native app of my game's latest development version. A bit cumbersome, so if AGS could spit out a native Mac app on its own? I can see value in that. Lots of freeware stuff released on the internet is unsigned, with the understanding for a Mac user to right-click & open instead. The xattr-cr is really something that seems to be required lately with Big Sur & M1 coming in.

ericoporto commented 3 years ago

First question to answer to supplying apple binaries for me is how does backward compatibility works. If I build something on Big Sur, does it run on High Sierra? Because we would be building binaries always on the latest version due to CI limitations.

What's the expectations on this matter? User wise it seems MacOS users are between 10.16 and 10.3 (bearing in mind this may be slightly skewed since it's a survey of Steam users) https://store.steampowered.com/hwsurvey/Steam-Hardware-Software-Survey-Welcome-to-Steam?platform=mac

Also no reasonable cloud service provides Apple silicon, so we currently could crosscompile for it (on Big Sur) but not run and verify it actually works.

About the wrapper, it's easy to build, it's just a directory with a specific structure and a xml file. The only thing is, like Linux, ags binary needs it's executable bit set, só when building from Windows we would need to package it in a tar.gz or a dmg or something that preserves/sets this bit.

hobbesjaap commented 3 years ago

True, I think quite a few apps extend back quite far in compatibility mode, it's more the future-proofing aspect of things that I was thinking about. As I mentioned on the forum, a few things I feel the MacOS port will need moving forward:

Now again, I'm well aware I'm making these comments in an almost flippant way since I know 0% about the work involved to make this happen. So the path to getting to these milestones can be very long indeed, but in order to futureproof the MacOS version of AGS, I think these two elements are key. So whilst the point of creating an M1 binary is not particularly relevant at this point, Apple is a company that's very happy to suddenly drop support for legacy hardware / software which makes it a bit of a pain for small-time open source projects like AGS. The upcoming move to SDL2 is already a great step towards futureproofing (for now) so perhaps there's opportunities to explore there?

I realise this thread is now at risk of veering significantly off-topic and raises potentially 2 feature requests. If desired, I could describe these in more detail in separate posts?

ericoporto commented 3 years ago

M1 -> it's just arm, we already have this.

Metal -> is already there through software renderer. A "proper" metal renderer would require someone not only writing one, but also maintaining it. So far no one has. I think a better strategy would be enhancing the SDL Renderer.

Better to split in separate topics.

Adding docs:

ericoporto commented 2 years ago

An year later, our CI is running on Big Sur and Monterey, so we can now actually cross compile on it to Apple Silicon. So supposing a binary built on these newer MacOS systems can run on an older OS version, this could be feasible.

Edit: 2023 update, CI is Apple Silicon now.

ericoporto commented 2 years ago

I just noticed that if someone unpacks it on Windows, set the files for their game and pack again it will be cursed by #471 too, because MacOS also requires ags binary to have executable flag set....

Edit: I actually found out there is a hacky file you can make on Windows that once it gets copy to a MacOS partition it's removed and used to set a particular file as executable but not sure yet this works in files inside .app terminated directories.

ericoporto commented 2 years ago

Hey, I will probably submit a PR at the weekend that does two things:

I may additionally reconfigure the build so that release and debug reuse the same machine (to reduce the chances the vm is unavailable), and also use ninja instead of make. The release will be generated for "older" available image from cirrus-ci.

In the future if someone wants to add MacOS building in the editor, the way to do so would be the following:

ivan-mogilko commented 2 years ago

Removed from 3.6.0 milestone; from what I understand few bigger changes have to be done first, so it's better to write task tickets based on this discussion and close this afterwards.

ericoporto commented 1 year ago

Minor update: I have no idea how to pick the oldest Xcode build that is the release build, and build JUST THAT on ags tag release, and also package JUST THAT in the packaging step of the CI. Paging @morganwillcock if you have any idea.

morganwillcock commented 1 year ago

I don't think it would be too much of a problem to select it, but I would worry what kind of guarantees are assumed or implied if the binary is provided with an AGS release.

  1. It won't be signed or notarized
  2. The version of Xcode used may prevent it running on older versions of macOS
  3. The version of Xcode will be raised by the CI provider and we wouldn't have a way to remain on an older version

I can't see it is feasible to make AGS releases that include the binary and also maintain support of it because we may not have a way to rebuild it.

ericoporto commented 1 year ago

The version of Xcode will be raised by the CI provider and we wouldn't have a way to remain on an older version

To be honest, Apple also raises it locally too, my Xcode doesn't even load if I don't update on Apple Store. The minimum current target right now on the latest is High Sierra - six years old. I believe everyone keeps up with Apple OS upgrades at least a bit. So, with latest tools, all below should be able to run a binary built with those

I would say providing a release experimentally would allow to advance the knowledge on macOS compatibility.

About signing and notarization it SHOULD NOT be signed or notarized, as we don't control the game that will be placed along.

morganwillcock commented 1 year ago

About signing and notarization it SHOULD NOT be signed or notarized, as we don't control the game that will be placed along.

From our perspective it should not be. From the perspective of someone downloading a game it should be. It is more a question of what providing the engine binary is implying in terms of continued support. The requirements to build the engine are a subset of the requirements needed to sign and notarize, so someone following Apple's guidelines already has everything they need. As an example, if macOS loses the ability to run unsigned binaries is that now an AGS bug or does that get ignored? If we are supplying binaries with no guarantees about where they run, does that also mean we should be including FreeBSD binaries with every release?

Do you want to just include the binaries in the GitHub release or would this get bundled with the Editor? Both are described in this issue. I think it would be fine to include it in the GitHub release and refer to it as experimental. I don't think it would be wise to try any type of integration with the Editor whilst the build process isn't reproducible. I did find these:

I'm sceptical on the legality, but maybe one of them provides a way to build that can work independently from the CI, without relying on one person's hardware.

ivan-mogilko commented 1 year ago

Do you want to just include the binaries in the GitHub release or would this get bundled with the Editor? Both are described in this issue.

IIRC I opened this issue with the intent to do the first, as people were repeatedly asking me how to run on mac, and I was not able to give them any simple download instructions. Which was a shame, since I knew that something is being built by our CI server.

Deploying the game for mac from the Editor should be a separate task imo. Maybe it will require additional steps, and adding dedicated set of preferences to the editor or project settings.

ericoporto commented 1 year ago

As an example, if macOS loses the ability to run unsigned binaries is that now an AGS bug or does that get ignored?

It can be considered that is "already lost" as is a significant effort to run something unsigned, but you can sign something unsigned, but you will sign the bundle. The bundle includes the game the user is creating and we don't have control on that.

If we are supplying binaries with no guarantees about where they run, does that also mean we should be including FreeBSD binaries with every release?

That is not the same case, in Xcode we can specify the minimum target and it will work, similarly on Android how we build with current tools and have limited compatibility with past Android versions.

In FreeBSD there is no such guarantee, it's more similar to how it works in Linux, and would need investigation on hoe whatever it's libc binary compatibility works.

Do you want to just include the binaries in the GitHub release or would this get bundled with the Editor?

Only with the GitHub Release, they are useless on the Editor as one would need an actual macOS to finalize the process (sign and notarization). Plus, it's best people distribute what they actually test - and yes, I have been testing the binaries we produce, and they so far have worked for the past years.

I'm sceptical on the legality, but maybe one of them provides a way to build that can work independently from the CI, without relying on one person's hardware.

Let's not do anything ilegal, our CI runs in actual Apple Silicon in actual macOS, lets use limit the scope to that.

morganwillcock commented 1 year ago

If you want to implement it, I think the best way would be to just use the hardcoded Xcode version number. The download is already available. When the CI provider changes the Xcode version, the CI failure will be a prompt to redo the release documentation to indicate which version of Xcode was used, and for someone to re-test it.

https://api.cirrus-ci.com/v1/artifact/build/$CIRRUS_BUILD_ID/build_macos/binaries/bin_14.0_release/ags