macOS release v3.0.3 is damaged

[xor-gate]

Just grabbed the latest release by hand, and it seems to be broken. Also the icon is a denied symbol. Running macOS Monterey 12.6 AMD64

[matty0ung]

Oh, that's odd. Do the previous releases work for you? Can you run an sha256 checksum of the file (per instructions at https://osxdaily.com/2021/12/17/check-sha256-hash-mac/)? If the checksum value you get isn't f450f050801da57b58107e7ae955235c50ac4cc00380792b34e25a5dce4602a2 then it would mean the file got corrupted.

[xor-gate]

If you mean the DMG it the same:

jerry@coconut ~ % shasum -a 256 Downloads/brewtarget_3.0.3_x86_64.dmg
f450f050801da57b58107e7ae955235c50ac4cc00380792b34e25a5dce4602a2  Downloads/brewtarget_3.0.3_x86_64.dmg

Something other is happening, maybe the app is not correctly signed (apple notarized)?

[matty0ung]

It's definitely not apple notarized. (AIUI you a paid developer account with Apple to notarize apps.) But I wouldn't expect that to stop you installing it - just to give an "untrusted developer" warning.

From your screenshot, am I right that you can mount the DMG OK and copy the app to the Applications folder? If so, if you run the app from the command line, does that give a more informative error message?

(Sorry for all the questions. I don't have a Mac, so I can't test any of this myself. The Mac builds are done on GitHub via GitHub Actions.)

[xor-gate]

I'm a developer and also know how macOS stuff works. Yes the dmg is mountable and the app is drag-drop-installable. As seen in the screenshot the "untrusted developer" warning is not there, so it must be something else why it is broken.

I did deep inspection of the .app bundle and it seems another .app bundle is within it for some odd reason.

jerry@coconut ~ % open /Applications/brewtarget_3.0.3.app/Contents/Resources/Brewtarget.app/Contents/MacOS/Brewtarget
The application cannot be opened for an unexpected reason, error=Error Domain=NSOSStatusErrorDomain Code=-128 "userCanceledErr" UserInfo={_LSLine=3863, _LSFunction=_LSOpenStuffCallLocal}

When I open the folder brewtarget_3.0.3.app/Contents/Resources/ in finder

Then it seems a bundle is within a bundle. Thats not how macOS distributes its software. Also as seen in the screenshot Finder/macOS is happy with a app icon:

Then with CTRL+click on the application the untrusted developer goes away and the app is made trusted.

It must still be approved in the Security & Privacy settings

As you can see I know how this can be a pain :-). I'm the developer of https://github.com/syncthing/syncthing-macos

Hope this helps others fix the problem

[xor-gate]

It is even worse. When approved and moved to /Applications it crashes...

As you can see it has something to do with a missing dynamic library:

Referenced from: '/Applications/Brewtarget
.app/Contents/Frameworks/libxalan-c.112.dylib'
Reason: tried: '' (no such file), '' (no such file), '' (no such file), '' (no such file), '' (no such file)

[matty0ung]

Thanks for the additional info. I'll play around with the CMake files to see what I can do.

[matty0ung]

FYI, I have the nested bundle problem solved (which was relatively easy) and am working on the dynamic libraries stuff. It's not a million miles away from what we're already doing for the Windows build, but it's different enough that I'm learning lots of new things about CMake...

[matty0ung]

Various build fixes in https://github.com/Brewtarget/brewtarget/pull/680 should make the Mac build less broken. The bundle passes the verify_app test in CMake but, of course, the acid test is to try it out on a real Mac.

[xor-gate]

I have downloaded the github actions build artifact from https://github.com/Brewtarget/brewtarget/actions/runs/3507082797 It seems to be still be broken on libxalan-c.112.dylib. Icon is also not displaying.

[matty0ung]

Thanks for the quick feedback. Will take another look.

Seems like libxalan-c.112.dylib and libxerces-c-3.2.dylib are getting included in Brewtarget.app/Contents/Frameworks (which is progress!) but not libxalanMsg.112.dylib. I thought I had fixed this, but obviously not! It may well be that I'm missing some vital step between creating the Brewtarget.app tree and transferring it onto the dmg file.

[matty0ung]

Closing now that release 3.0.4 is out, but please re-open if there are still problems. (I don't have a Mac so I can't do as much testing of this as I would like.)

[xor-gate]

Stil broken

Termination Reason:    Namespace DYLD, Code 1 Library missing
Library not loaded: '/usr/local/opt/xerces-c/lib/libxerces-c-3.2.dylib'
Referenced from: '/Applications/Brewtarget.app/Contents/Frameworks/libxalanMsg.112.dylib'
Reason: tried: '/usr/local/opt/xerces-c/lib/libxerces-c-3.2.dylib' (no such file), '' (no such file), '' (no such file)
(terminated at launch; ignore backtrace)

[matty0ung]

Ah, OK, thanks. Sorry this is turning out to be a bit of a saga. I'll get back to the drawing board. (I've been having so much "fun" with CMake and CPack that I'm close to taking a different approach to build and packaging. We'll see.)

[xor-gate]

Distributing a C++ application in a cross platform way is a pain. I know, even with CMake it is hard.

[matty0ung]

Part of my investigations are looking at the Qt tool that's supposed to make it easier (Qt Installation Framework). It's not without it's own foibles, but, if I can get it to play ball, I'm hoping it would mean there's less divergence between what we have to do for the different platforms.

[xor-gate]

Yeah but if the project uses cmake/cpack, I would not recommend to add another approach to packaging as nobody can test it properly. There must be a way to learn cpack to package it correctly. I found also this cmake module https://cmake.org/cmake/help/latest/module/BundleUtilities.html. I think the problem is not all libraries are packed into the bundle. And the executable rpaths must be rewritten to use the bundle dylib (so/dll) files.

[matty0ung]

Oh yes, I know about BundleUtilities, and its rather patchy documentation. You can see my comment in CmakeLists.txt for some of the fun I had getting fixup_bundle and verify_app to run. You can also see explanations of my attempts to put the right libraries in the right place in the bundle (clearly unsuccessfully). I even found out how to examine the contents of a Mac bundle on Linux (see end of comment here).

It may be that we need someone who actually owns a Mac to work out what the correct CPack incantations are...

[matty0ung]

There must be a way to learn cpack to package it correctly.

Yes, I used to think this, about 50 commits ago. :smile_cat:

if the project uses cmake/cpack, I would not recommend to add another approach to packaging

Agreed. If we were to switch to something else it would include changing the way we build on all platforms. I've been a bit reluctant to do this for a 3.0.x release, but I do already have all the compilation and unit tests running in a Meson build for instance. It's faster than CMake and the documentation is, IMHO, better. Meson won't do the packaging but that might not be a bad thing if we can use the Qt tool instead (and get Meson to invoke the Qt tool).

Just saying that, although my preferred approach for 3.0.x is to get a fix to CMake/CPack, if that turns out not to be doable, then there is a Plan B.

[mattiasmaahl]

Agreed, trying to fix cmake for now is, IMHO, the way to go. I don't have any experience with Meson so I cannot weigh in on that, will have to look into what it is and what ot can do before i can join a discussion.

[matty0ung]

So, I had another go with https://github.com/Brewtarget/brewtarget/releases/tag/v3.0.5. This time my approach was to make more of the libraries static, so we shouldn't need to have to find them in the bundle. Of course, it may be that this did not work. I'll leave this ticket open until someone with a Mac has a chance to give us feedback on the latest release.

Happy Christmas either way! :christmas_tree: :santa:

[eli-schwartz]

It is technically possible to build a static Qt, and then build+link projects to that, which simplifies deployment even more.

It doesn't work if you need WebEngine, but you don't use that anyway.

[matty0ung]

It is technically possible to build a static Qt, and then build+link projects to that, which simplifies deployment even more.

You're right. I did have a quick look at this before and it seemed like it was going to require building Qt from source, which sounded a bit painful (eg because it would rather slow down the automated GitHub builds we use). But if it ends up being what's needed, then I guess we'll have to bite the bullet.

[eli-schwartz]

If you end up doing that, you should absolutely build it once and upload it as a precompiled binary artifact which you'd then download in automated builds. ;)

[xor-gate]

Having all opensource libraries staticly linked is mostly a license violation, as the licenses don't have a "static linking exception". One must investigate why dynamic linking doesn't work correctly. This is mostly due to the macOS executable rpath to libraries not correctly set.

[matty0ung]

Having all opensource libraries staticly linked is mostly a license violation, as the licenses don't have a "static linking exception". One must investigate why dynamic linking doesn't work correctly. This is mostly due to the macOS executable rpath to libraries not correctly set.

In our particular case, I don't think that's an issue. https://xerces.apache.org/xerces-c/faq-distrib-3.html says "You can also link your application to a static version of the Xerces-C++ library in which case you won't need to distribute any extra libraries". Xalan-C++ is distributed under the same license (Apache Software License version 2.0).

[95ep]

So, I had another go with https://github.com/Brewtarget/brewtarget/releases/tag/v3.0.5. This time my approach was to make more of the libraries static, so we shouldn't need to have to find them in the bundle. Of course, it may be that this did not work. I'll leave this ticket open until someone with a Mac has a chance to give us feedback on the latest release.

Happy Christmas either way! 🎄 🎅

I've tested the latest, 3.0.6, build and unfortunately I still get the same error_

Termination Reason:    Namespace DYLD, Code 1 Library missing
Library not loaded: /usr/local/opt/xerces-c/lib/libxerces-c-3.2.dylib
Referenced from: <90557DC4-A1E4-3E01-A07D-8F144164D6CF> /Applications/Brewtarget.app/Contents/Frameworks/libxalanMsg.112.dylib
Reason: tried: '/usr/local/opt/xerces-c/lib/libxerces-c-3.2.dylib' (no such file), '/System/Volumes/Preboot/Cryptexes/OS/usr/local/opt/xerces-c/lib/libxerces-c-3.2.dylib' (no such file), '/usr/local/opt/xerces-c/lib/libxerces-c-3.2.dylib' (no such file), '/usr/local/lib/libxerces-c-3.2.dylib' (no such file), '/usr/lib/libxerces-c-3.2.dylib' (no such file, not in dyld cache)
(terminated at launch; ignore backtrace)

I don't have any experience with building Mac apps myself, but happy to help out if logs or similar is needed

[matty0ung]

Thanks for the feedback. I have been working on a new way of doing the packaging that, amongst other things, I hope will help us solve this problem. Will definitely be grateful for more Mac install testing soon. Watch this space...

(Essentially our current approach of using CPack is painful to debug because there is so much automagical logic going on under the hood that it can be hard to know settings you need to change to get an individual file placed in a different directory. The new approach of controlling things with a Python script is a bit less "clever" but should be easier to fine-tune. It already helped us solve some problems with the Windows packaging, so I'm keeping my fingers crossed it can do a similar service for the Mac package.)

[xor-gate]

Packaging software is a craft when doing it for multiple operating systems. I can understand when you don't own the target system it is difficult to pack it correctly. CMake can be a little pain to get right, but making your own scripts to package to "finetune" it better can get you trapped in making the same mistakes and learnings. I'm an experienced macOS user (and open-source developer). But I cant make time to help you out to fix it as having other priorties. For brewing I use the Brewer's Friend Complete Beer Recipe Builder and then write them down for archive on my blog. I have used brewtarget in the past on macOS but it screwed up the database and had no backup :-(.

What I can offer is some advice, and test the version on my macOS system(s). Good luck!

[matty0ung]

Thanks for sharing your thoughts.

The 3.0.7 release has the latest attempt at packaging everything up for Mac. If you get a moment to test it for us on a real Mac, that would be much appreciated. I'm not 100% sure it's going to work, but I think we're moving in the right direction!

Sorry about your bad experience with Brewtarget in the past. If you do ever want to try it again for your brewing, we do now have an automated rolling back-up of the database file, so it should be a lot less likely that you lose data.

[95ep]

Downloaded and tested 3.0.7 now. This time it's a new library causing the crash:

Termination Reason:    Namespace DYLD, Code 1 Library missing
Library not loaded: @rpath/libxalanMsg.112.dylib
Referenced from: <FFEB71DC-F4FE-399D-A722-6B4D1DB43779> /Applications/brewtarget_3.0.7.app/Contents/Frameworks/libxalan-c.112.dylib
Reason: tried: '/Applications/brewtarget_3.0.7.app/Contents/Frameworks/../lib/libxalanMsg.112.dylib' (no such file), '/System/Volumes/Preboot/Cryptexes/OS@rpath/libxalanMsg.112.dylib' (no such file), '/Applications/brewtarget_3.0.7.app/Contents/Frameworks/../lib/libxalanMsg.112.dylib' (no such file), '/usr/local/lib/libxalanMsg.112.dylib' (no such file), '/usr/lib/libxalanMsg.112.dylib' (no such file, not in dyld cache)
(terminated at launch; ignore backtrace)

On another note, in this release the logo is back as application icon

[matty0ung]

Thanks for trying this. Sorry we're not quite there yet! That error message should be helpful though...

[matty0ung]

Another package for folks to try: https://github.com/Brewtarget/brewtarget/releases/tag/v3.0.8

If that doesn't work, I have an additional tweak that may help.

[95ep]

Looks like it works! Successfully installed the 3.0.8 release and imported my old db.

[matty0ung]

Looks like it works! Successfully installed the 3.0.8 release and imported my old db.

Excellent news!

We'll stick with the new Meson build + Python scripts for doing the release build and packaging. (It's actually the Qt tool macdeployqt that does most of the work on the packaging, just as when we were trying to use CPack. The difference is that, with the Python script, it's easier to do the extra "tweaks" for the things that macdeployqt can't quite handle.)