ChordPro / chordpro

Reference implementation of the ChordPro standard for musical lead sheets.
Other
308 stars 51 forks source link

Build wxchordpro from source on macOS ARM #421

Open Desbeers opened 2 weeks ago

Desbeers commented 2 weeks ago

It is possible but not so easy...

System: macOS Sonoma on a M1 MacBook Pro

Related: I had no problems at all building xwchordpro on my Fedora 40 box straight from the source.

Perl version

I use system-perl. I don’t have any preference for the perl-version (tried homebrew and perbrew as well) but noticed that many script have #!/usr/bin/perl at the top; preventing alternative Perls??

wxWidgets

I did build xwWidgets from source:

https://forum.wxwidgets.org/3.2/plat_osx_install.html

The demos are working well and ARM native.

Only thing I had to do differently is build with...

` ../configure --enable-debug --with-libtiff=builtin

Or else it did not compile.

See https://github.com/cjcliffe/CubicSDR/issues/838

While not recommended; I did a sudo make install anyway :-)

cpanm

I did install cpanm with homebrew Doing sudo cpanm Alien:: wxWidgets works without errors. sudo cpanm Wx gives an error:

No such 'link' library: 'adv' at /Users/Desbeers/.cpanm/work/1725099354.36136/Wx-0.9932/build/Wx/build/MakeMaker.pm

Did a lot of Googeling and this was the most interesting:

https://sources.debian.org/patches/libwx-perl/1:0.9932-8/

xwPerl

I download the xwPerl source and applied all Debian patches:

patch -p1 < *.patch

Running perl Makefile.PL works now Running make resulted (after a while) with an error

DND.c:453:15: error: incompatible integer to pointer conversion assigning to 'NativeFormat' (aka 'const __CFString *') from 'wxDataFormatId'

I just removed the offending code from DND.c (in 2 places); compiled and it worked.

Did a sudo install.

ChordPro source

pp/macos/wxchordpro_pp.pl

I had to change the paths in this file because homebrew installs its stuff in a different location on ARM Macs.

Went to chordpro\pp\macos and did a make

YEH!!!! It compiled!

But did not run...

Termination Reason:    Namespace DYLD, Code 1 Library missing
Library not loaded: @executable_path/libwx_osx_cocoau_xrc-3.2.dylib
Referenced from: <DFABB10E-63B6-3A0E-B32F-BE8A050328A9> /Users/USER/Desktop/ChordPro.app/Contents/MacOS/dndhandler
Reason: tried: '/Users/Desbeers/Desktop/ChordPro.app/Contents/MacOS/libwx_osx_cocoau_xrc-3.2.dylib' (no such file)

libwx_osx_cocoau_xrc-3.2.dylib is not in the package.

libwx_osx_cocoau_xrc-3.2.0.2.3.dylib is...

There is a libwx_osx_cocoau_xrc-3.2.dylib in /usr/local/lib but that is a symlink.

Same is for all the other .dylib’s.

Looks like the Alien::xwWidgets is passing the symlink and not the actual lib in its config...

(/Library/Perl/5.34/darwin-thread-multi-2level/Alien/wxWidgets/Config/mac_3_2_5_uni_nc_0.pm)

So; I changed all the paths in that file...

Compiled again; but it still didn’t run...

EXC_BAD_ACCESS (SIGKILL (Code Signature Invalid))

Well; that one is familiar to me... ARM code must be signed. Moved the app from the disk-image to my Desktop; removed stuff from its root (besides icons some other .dSYM stuff??) and resigned it...

Didn’t work...

Found out I have to move it to a place that is not part of the iCloud (my Desktop folder is) to resign... (go figure...)

Tried again to resign...

RUNNING!!!!!

Issues to solve

Conclusion

My biggest issue was getting wxPerl up-and-running. It seems unsupported.

Having Classic natively running on ARM was not my main goal. I just wanted to be able to build it and see if I can get it a bit better for old Macs or maybe other plafroms. My SwiftUI wrapper is the way forward but while Classic is classic on macOS, it is not on Linux and Windows.

Desbeers commented 2 weeks ago

Well, I don’t want to be too optimistic; but I bluntly removed all macOS constraints from the Wx code and everything seems to work just fine.

All preferences where saved an still there after a restart.

Only left the initial ‘open dialog’ out because that crashed the application. Selecting a file or just cancel the dialog didn’t work.

Desbeers commented 2 weeks ago

(last time I patched a source like this when I wanted to get my Sound Blaster AWE32 fully working; the Linux kernel pre version 2 somewhere in 1996. I feel a bit old but its funny I still remember it and never expected that the start of my Open Source journey gave me useful knowledge thousand years later. I don’t mind to fight compile problems; part of my hobby. I -do- mind commercial restrictions and currently living in a doubtful world between macOS and Linux)

sciurius commented 2 weeks ago

Apparently, much has changed the last couple of years. When I first tried to port wxPerl to MacOS I needed to figure out everything myself. There were no patches at the time. When I got to the point that something was running I was already very glad.

It looks worth the efforts to try a new port or wxPerl, using the debian patches. It could lead to a less crippled 'classic' version. Unfortunately I cannot use system perl on Catalina. It is too old (5.18; ChordPro requires at least 5.26). But for the time being I assume the cripples are due to my incomplete port of wxPerl only.

Desbeers commented 2 weeks ago

When I first tried to port wxPerl to MacOS I needed to figure out everything myself.

My upmost serious complements you did get it up and running! Must have been a lot of work...

Well, the saga continues... My build did not run on Monterey:

Screenshot 2024-09-01 at 13 18 14

Fixable by providing the minimum OS version in the pp/macos/MakeFile

${CXX} -mmacosx-version-min=10.15 -DMACOS -o "${DEST}/dndhandler" \

Now it starts but crashed:

Termination Reason:    Namespace DYLD, Code 1 Library missing
Library not loaded: '/opt/homebrew/*/libpcre2-32.0.dylib'
Referenced from: ''
Reason: tried: '/opt/homebrew/*/libpcre2-32.0.dylib' (no such file), '' (no such file), '' (no such file)Library not loaded: '/

(note that Referenced from is empty now and the library does not contains @executable_path/)

Did not find a solution yet and I don’t know if the issue is in my wxWidgets build or my wxPerl build...

Just guessing but it seems some wx related library is internally hardcoded to use the /opt/homebrew stuff...

I suggest not to touch you Catalina installation; treasure it!! :-)

To be continued; well worth the effort! Not just for classic.

Desbeers commented 2 weeks ago

I must admit that wxWidgets deserved more appreciation from me than in my initial opinion and I wished I spend some more time with it instead of just writing a brand-new wrapper :-)

Desbeers commented 2 weeks ago

It looks worth the efforts to try a new port of wxPerl, using the debian patches.

It would be great if there is going to be a supported wxPerl again instead of having Linux distributions patching it to keep it alive and some random macOS users (like me) spending days to compile it and having CPAN completely broken...

How do you get it to work on Windows? It must be a miracle! :-D

sciurius commented 2 weeks ago

It's a bit of a story...

Windows users kept asking for a GUI, so I needed to provide at least a basic GUI wrapper. I immedeately choose for wxWdigets, since it would provide native looking applications on all three (for me, at least) major platforms: Linux, Windows and Mac, even though the target was Windows, I could develop on Linux.

At that time I was working with Citrus Perl, a custom perl distribution that had Wx integrated (and a lot more, like databases). It also had a nice (but proprietar) packager, Cava Packager, to distribute standalone kits with your applications. Citrus Perl and Cava Packager were developed by Mark Dootson, who also did most of the wxPerl development. He was a great help in building my Wx applications. We were in the process of open sourcing Cava when Mark suddenly stopped all his wxPerl, Citrus and Cava activities for unknown reasons.

Anyway, by that time I had a simple wrapper around one of my other applications (EekBoek, a bookkeeping program) and I basically copied that approach for ChordPro. Nothing fancy, just something a GUI user could use to access ChordPro from the desktop. I hoped some enthousiast musician with programming skills would eventually jump in to create a real GUI app.

That did not happen. ChordPro users seem to be musicians and not programmers ☹. So I needed to take up the burdon of maintaining and expanding wxChordPro on Windows and, later, Mac, platforms that I do not use myself.

Due to the demise of Citrus Perl I switched to Strawberry Perl and built wxWdigets and wxPerl myself. Building wxWidgets was straightforward since it is officially maintained for Windows, but wxPerl required a couple of trivial changes due to the newer perl version. I adopted (and augmented) the 'ppl' approach which turned out to work nicely, InnoSetup took care of creating the installer package.

On MacOS I experimented with system perl first, but that turned out not to be feasible. I switched to brew perl. Luckily, brew also had a package for wxWidgets. (3.0 at the time). Porting wxPerl turned out not to be that hard, since there were a number of patches for 3.0. Unfortunately, the result was slightly crippled. Much later I upgraded to wxWidgets 3.2. This time I needed to find out the changes required for wxPerl myself since there were no patches available. With a little help of the wxWdigets community I could get it to build and run. With a dedicated drag-n-drop handler it at least looked a bit like a Mac application. But it was still crippled.

That's where you jumped in... Finally a musician with programming skills.

sciurius commented 2 weeks ago

Fiddled with reloc.pl (it have an exception for ‘libpcre2’ for some reason?)

I think because the library name of libpcre2 didn't obey the pattern of the other library names.

sciurius commented 2 weeks ago

Well, I don’t want to be too optimistic; but I bluntly removed all macOS constraints from the Wx code and everything seems to work just fine.

IIRC, the problems were that the macOS system calls that wxWidgets (at the time) used were not functional in sandboxed context. There were other system calls that I needed using but these were not part of wxPerl.

sciurius commented 2 weeks ago

I bluntly removed all macOS constraints from the Wx code and everything seems to work just fine.

Can you verify that loading a file into the editor does not change programming quotes to typographical quotes?

sciurius commented 2 weeks ago

A damper to the party joy ...

I retrieved the instructions and patches that I used for building wxPerl, and it turns out that most of the debian patches are just generic wxPerl - wxWidgets 3.0/3.2 patches and, as such, have already been included in the current build.

Attached is one additional patch that I needed to apply.

wxWidgets_3.2_more.patch.txt

sciurius commented 2 weeks ago

I bluntly removed all macOS constraints from the Wx code and everything seems to work just fine.

I did the same on Catalina. The result was crippled. Open dialogs (file open, preferences, ...) do not appear. File save turned out to be possible.

Then I copied the same dmg to Big Slur. Everything seems functional. Even the initial open dialog 8).

So the cripples are not in ChordPro, wxPerl, wxWidgets but must be in some other layer underneat. Any idea?

I could make the crippling dependent on OS version...

Desbeers commented 2 weeks ago

Can you verify that loading a file into the editor does not change programming quotes to typographical quotes?

Here we go again... :-) It does change them... Looks like I have to apply your patch to wxPerl and recompile... Saw the notes in the source. I’ll try.

Desbeers commented 2 weeks ago

So the cripples are not in ChordPro, wxPerl, wxWidgets but must be in some other layer underneat. Any idea?

Not really... Just double-checked and the app is not sandboxed. Looked into the wxWidgets code for the file dialog (src/cocoa/filedlg.mm) and it is just an ordinary NSOpenPanel attached to the current window. It opens it with beginSheetModal; nothing special. I can only speculate that wxWidgets might not be able to find the parent window on Catalina...

I assume the initial open dialog is also not shown on Catalina?

File save turned out to be possible.

Did you just saved an existing file on Catalina (that does not show a dialog) or tried to save a new file (that should show a dialog)? Just to check if dialogs can be visible at all...

Desbeers commented 2 weeks ago

Attached is one additional patch that I needed to apply.

Applied the patch; recompiled xwPerl, recompiled ChordPro, and....

The quotes are not changed anymore :-)

Desbeers commented 2 weeks ago

Then I copied the same dmg to Big Slur. Everything seems functional. Even the initial open dialog 8).

Tried one more time to enable the initial dialog on Sonoma but no... But I managed to grab the error:

Terminating app due to uncaught exception 'NSInternalInconsistencyException', reason: 'NSApp with wrong _running count'

Did my usual Googeling...

https://github.com/fyne-io/fyne/issues/782

According to the macOS documentation it is when an app is launched more than once (which is not supposed to happen on macOS).

Anyway, I can’t be bothered about it too much now but I think it might related to the Catalina NSOpenPanel issue. There is no Window available and it looks to me wxWidgets is just restarting itself in this case on Sonoma and that is not allowed... Just guessing...

To quote from the same issue:

Each major macOS update breaks things for us somehow :(

Oh well... Every cross-platform UI seems to struggle on macOS ;-)

sciurius commented 2 weeks ago

I assume the initial open dialog is also not shown on Catalina?

When I try to display the initial open dialog I get a spinning ball and basically have to reboot.

So you think it is a matter of the (other) dialogs not displaying? That should be solvable.

Did you just saved an existing file on Catalina (that does not show a dialog) or tried to save a new file (that should show a dialog)? Just to check if dialogs can be visible at all...

Saved an existing file. There is no Save As in the File menu.

sciurius commented 2 weeks ago

wxWidgets is just restarting itself in this case

That could explain the unbreakable spinning ball.

Desbeers commented 2 weeks ago

Saved an existing file. There is no Save As in the File menu.

A new file should trigger the dialog when you change its content and close the window... It does for me.

sciurius commented 2 weeks ago

Oh yes, the question "do you want to save your changes" appears, but clicking [Yes] does not result in a (new) file selection dialog. Consistent with the other file dialogs that do not show.

sciurius commented 2 weeks ago

These are the errors I get:

2024-09-02 17:49:04.385 wxchordpro[5202:25531] +[NSXPCSharedListener endpointForReply:withListenerName:]: an error occurred while attempting to obtain endpoint for listener 'com.apple.view-bridge': Connection interrupted

Desbeers commented 2 weeks ago

Oh yes, the question "do you want to save your changes" appears, but clicking [Yes] does not result in a (new) file selection dialog. Consistent with the other file dialogs that do not show.

Now we get somewhere!

So you think it is a matter of the (other) dialogs not displaying? That should be solvable.

Everything is solvable when the source code is open :-)

My first priority is to get a build that is not insisting on my local installed '/opt/homebrew/*/libpcre2-32.0.dylib’. Any hints?

Then I will look into the dialog issue; I have some ideas. Problem is, Swift and SwiftUI are like the Dutch language to me, native. Perl, CC, Objective stuff; those languages are like German to me. I can understand it more or less but I can’t speak or write it :-D

Desbeers commented 2 weeks ago

Finally a musician with programming skills.

You have no fking idea how proud I am to be called a musician* with programming skills* :-)

My Telecaster is looking in love to me again and forgave me my hours of programming...

sciurius commented 2 weeks ago

My first priority is to get a build that is not insisting on my local installed '/opt/homebrew/*/libpcre2-32.0.dylib’. Any hints?

reloc.pl gives messages like these:

+ install_name_tool -change "/usr/local/opt/pcre2/lib/libpcre2-32.0.dylib" "@executable_path/libpcre2-32.0.dylib" "ChordPro.app/Contents/MacOS/libwx_osx_cocoau_xrc-3.2.dylib"

Could it be that the relocs fails to change one of the libraries?

Desbeers commented 2 weeks ago

It should give messages for all but it does not for non-WX stuff.

perl ./reloc.pl ChordPro.app/Contents/MacOS/libpng16.*.dylib
perl ./reloc.pl ChordPro.app/Contents/MacOS/libjpeg.*.dylib
perl ./reloc.pl ChordPro.app/Contents/MacOS/libtiff.*.dylib
perl ./reloc.pl ChordPro.app/Contents/MacOS/libz*.dylib
perl ./reloc.pl ChordPro.app/Contents/MacOS/libpcre2-32.*.dylib
perl ./reloc.pl ChordPro.app/Contents/MacOS/libwx*.dylib
+ install_name_tool -id "@executable_path/libwx_baseu-3.2.0.2.3.dylib" "ChordPro.app/Contents/MacOS/libwx_baseu-3.2.0.2.3.dylib"
(etc)

WX stuff is moved with a message. All the rest is just ignored? If I rename something to something that does not exists it errors so the file is found.

sciurius commented 2 weeks ago

perl ./reloc.pl ChordPro.app/Contents/MacOS/libpcre2-32.*.dylib + install_name_tool -id "@executable_path/libpcre2-32.0.dylib" "ChordPro.app/Contents/MacOS/libpcre2-32.0.dylib"

sciurius commented 2 weeks ago

What is the full name of the pcre2 lib?

Desbeers commented 2 weeks ago

libpcre2-32.0.dylib, just like yours

The lib(image).dylib are not moved either but not needed on Sonoma.

sciurius commented 2 weeks ago

Full path? output of otool -L fullpathto/libpcre2-32.0.dylib ?

Somehow

perl reloc.pl fullpathto/libpcre2-32.0.dylib

doesn't seem to recognize.

Desbeers commented 2 weeks ago
Desbeers@UberBook ~ % otool -L /opt/homebrew/lib/libpcre2-32.0.dylib
/opt/homebrew/lib/libpcre2-32.0.dylib:
/opt/homebrew/opt/pcre2/lib/libpcre2-32.0.dylib (compatibility version 14.0.0, current version 14.0.0)
/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1345.100.2)
sciurius commented 2 weeks ago

reseq.pl veronderstelt alle libraries in dezelfde locatie als wxwidgets.

Desbeers commented 2 weeks ago

I'm afraid it is symlinking....

sciurius commented 2 weeks ago

Is there a reason you didn't install wxwidgets from homebrew?

Desbeers commented 2 weeks ago

Yes, it did crash any attempt to build ChordPro. Not the building; the running...

Desbeers commented 2 weeks ago

/opt/homebrew/lib/libpcre2-32.0.dylib is a symlink....

To be continued; good night!

sciurius commented 2 weeks ago

Hmm. You could try changing reloc.pl adding a line

  my $prefix = Alien::wxWidgets->prefix;
  my @libs = Alien::wxWidgets->shared_libraries("core");
+ $prefix = "";
  die("Cannot find libs version\n") unless $libs[0] =~ /-([0-9._]+)\.dylib/;
sciurius commented 1 week ago

Yes, it did crash any attempt to build ChordPro. Not the building; the running...

The only reason for crashes I can think of is library mismatches. The perl packager ('pp' in the Makefile) collects all necessary libraries but it is very well possible that it misses one, or picks a library from the wrong source. Or, as currently happens in the non-homebrew build, fails to correctly relocate some libraries.

Another thing is pollution, e.g. left overs from earlier experiments that cause the wrong libraries to be included. That's the main reason I work with virtual systems, since they can be (in my setup, are always) restored to the initial state after experimenting.

Desbeers commented 1 week ago

Hmm. You could try changing reloc.pl adding a line

  my $prefix = Alien::wxWidgets->prefix;
  my @libs = Alien::wxWidgets->shared_libraries("core");
+ $prefix = "";
  die("Cannot find libs version\n") unless $libs[0] =~ /-([0-9._]+)\.dylib/;

This did the trick! My build is movable!

The log is also much longer now.

While I'm happy this works; I cannot really understand why... Can you explain?

Desbeers commented 1 week ago

Another thing is pollution, e.g. left overs from earlier experiments that cause the wrong libraries to be included. That's the main reason I work with virtual systems, since they can be (in my setup, are always) restored to the initial state after experimenting.

I should do the same; my Mac is a mess... Think I’m going to do a clean install and use VM’s as well. Already have it for Monterey and see if I can build one for Catalina as well so I can try Intel stuff again on my Mac M1...

sciurius commented 1 week ago

While I'm happy this works; I cannot really understand why... Can you explain?

Well... reloc.pl is really a quick hack that turned out to be effective. In my case ☺.

The idea is that for all dylibs included in the kit, the paths to the (also included) dylibs are made relative using @executable_path. Somehow I felt the need to restrict the relocation to libraries that were part of the locally built libs in /usr/local. In your case, you also have libs from /opt/homebrew.

The latter is surprising, since you are not using homebrew stuff. Or do you?

Desbeers commented 1 week ago

The latter is surprising, since you are not using homebrew stuff. Or do you?

I do use homebrew. My /usr/local/lib is actually a symlink to /opt/homebrew/lib and that dir is full with symlinks to all the .dylib stuff, except for the libwx_ stuff that I compiled myself; those a real* files...

It’s a mess; I have to do some housekeeping first :-)

sciurius commented 1 week ago

I'm quite confident that after a thourough cleanup the homebrew wxwidgets build will work...

Desbeers commented 1 week ago

I'm quite confident that after a thourough cleanup the homebrew wxwidgets build will work...

Yes, it did! Started from scratch in a new VM. Decided to go for a ‘full’ homebrew approach.

Found out how to use homebrew Perl, finally.. Catalina does not contains pp pre-installed (part of PAR::Packer); newer versions do, and it is in usr/bin/, insisting on ‘system’ Perl... Installing PAR::Packer and have my own stuff in front of the path solved that problem.

Also found out how to use cpanm without sudo; not throwing its stuff all around the system :-)

Still needed to adjust the paths in wxchordpro_pp.pl for ARM of course but that is just a detail that should be easy to fix? Just looking at brew --prefix maybe? Mine are:

# Explicit libraries.
--link=/opt/homebrew/lib/libpng16.16.dylib
--link=/opt/homebrew/lib/libjpeg.8.dylib
--link=/opt/homebrew/lib/libtiff.6.dylib
--link=/opt/homebrew/Cellar/zlib/1.3.1/lib/libz.1.3.1.dylib
--link=/opt/homebrew//lib/libzstd.1.dylib
--link=/opt/homebrew//lib/libpcre2-32.0.dylib

And code-signing and setting the minimum wxWidgets version in pp/macos/Makefile. I’ll send you a PR for that; its trivial and should not break anything.

Last, reloc.pl is not insisting on /usr/local but using the result of Alien::wxWidgets->prefix. That is /opt/homebrew in my case, so correct.

reloc.pl renames $name near the end to a shorter version; breaking the relocation. I just removed the renaming and all seems to work well.

I’m going to reset my VM and do this one more time and make some more notes...

Good progress!

Desbeers commented 1 week ago

I’m a bit unhappy about the fact that my new macOS wrapper is living in a completely different world than the other wrappers. That is one of the reasons I spend so much time on WX now. I would love to add an option to the GUI to export a folder into a beautiful PDF songbook. ChordPro can do that.

I have it in my Chord Provider application and its easy for me to add it to the ChordProMac wrapper but I’m hesitating to do that; it will make it even more Alien::SwiftUI (haha!).

Screenshot 2024-09-03 at 21 39 17

Maybe I should just let-go my not 100% native principles...

Desbeers commented 1 week ago

My Kodi server is a Mac Mini running Catalina.... I build wxWidgets from source; including the demo’s and samples...

The samples folder contains an dialog app.

I does open all dialogs; including the file open/save dialogs.

So, it looks to me the problem is in xwPerl...

sciurius commented 1 week ago

It's even worst than that...

If you install Wx::Demo and run the demo program (attached) you can choose the file and dir pickers from wxPerl > Controls > Picker Controls and they are fully functional... So the real problem may be ChordPro related -- somewhere in the glue (dndhandler) or packaging.

wxdemo.pl.txt

sciurius commented 1 week ago

Working on folder export (bugs found: 2, bugs in wxGlade: 1, new sensational feature: 1).

Desbeers commented 1 week ago

It's even worst than that...

I don’t consider it worse; it brings us closer to the source of the problem!

Just had a quick look at dndhandler.cpp; it has the static char selfpath[PATH_MAX]; stuff like recently fixed in pp/linux/ppl.c with _NSGetExecutablePath... Could be related?

Also maybe; I bluntly removed code from DND.c in the xwPerl source but you didn’t so I don’t think it is related...

Stupid question: how to I build or run ChordPro with/in DEBUG? So I see the logs printed in #ifdef DEBUG??

Desbeers commented 1 week ago

If you install Wx::Demo and run the demo program

Work fine here (and pretty impressive) on my Sonoma and gives me also some nice source-code to study...

I’m sure we’ll get Classic in a reasonable state!

sciurius commented 1 week ago

When removing all cripples I can run

perl -ICPAN -Ilib script/wxchordpro.pl 

Fully functional except for the initial opening dialog. When I press [OK] after selecting a filename, it crashes with:

2024-09-04 19:08:24.639 perl[1901:37824] *** Terminating app due to uncaught exception 'NSInternalInconsistencyException', reason: 'NSApp with wrong _running count'

Same results after building:

cd pp/macos
build/ChordPro.app/Contents/MacOS/wxchordpro file.cho

and from the dmg (after move to applications):

/Applications/ChordPro.app/Contents/MacOS/wxchordpro file.cho

As long as I do not touch the initial opening dialog everything seems functional.