search

ChordPro / chordpro

Reference implementation of the ChordPro standard for musical lead sheets.
Other
310 stars 51 forks source link

MacOS rewrite #373

Open sciurius opened 4 months ago

Desbeers commented 4 months ago

As I wrote to the list, I volunteer to update the current macOS wrapper for ChordPro.

I wrote a proof of concept macOS SwiftUI wrapper:

https://github.com/Desbeers/ChordProMac

For now, it is just calling the official binary to do the hard work:

/Applications/ChordPro.app/Contents/MacOS/chordpro

My main goal for now was to bridge SwiftUI with the core ChordPro binary.

That is working now, so adding more arguments like settings to the core is not that hard to do.

A harder struggle will be to include the core into the macOS package itself but I’m sure that will be doable. Something new for me, but I’m sure we find out how to do!

Eventually, I would love to have this wrapper in the official ChordPro source! This is not meant to be an alternative implementation :-)

I try to keep is a simple as possible. No external dependencies and try to support older macOS versions as best as I could.

sciurius commented 4 months ago

Looks great already! Stupid question of a typical non-Mac user: How do I run it?

Desbeers commented 4 months ago

I assumed you have a Mac and Xcode installed.

Else, helaas-pindakaas... You need a Mac to build and run it...

Do you have a Mac? Since you provide a binary for macOS that was obvious to me...

sciurius commented 4 months ago

No, I do not have a Mac... I have a VBox with MacOS that I use to make the MacOS binary distributions, but it doesn't have Xcode. But if I can make binary distributions I assume you can? Maybe start with the ChordPro dmg and add your stuff to it?

Desbeers commented 4 months ago

I can move your binary into the macOS package (after some homework) and build it.

Hoverer, that makes it mine, not yours.

That is not the goal. You should build it somehow because I don’t want to be a a new implantation :-)

sciurius commented 4 months ago

First step is to make it work and in such a shape that an average MacOS user will like it.

In an ideal situation all binary kits should be built centrally and automatically, but that isn't the case anyway. Kits for Debian and Fedora are built indepently. I don't know if GitHub can build MacOS kits, if so that would be an option. For the time being it is perfectly acceptable to roll out a release (source) and that others (me included) build the binaries.

Desbeers commented 4 months ago

I’m unable to build the Mac dmg myself for now, it has too many missing packages to compile, wrong perl version etc... However, I just moved the precompiled ‘chordpro’, ‘libperl’ and ‘lib’ + ‘script’ folders from the official app into my source and the result is a self-containing app that seems to work well.

I don’t have an Apple developers account so I’m unable to make a pre-compiled app to share.

It would be great if somebody with a Mac can download the source and compiles it with Xcode so I know it’s working as I intended.

sciurius commented 4 months ago

Isn't it possible to add your app to the prebuilt dmg and use it as entry point instead of dndhandler?

I build the MacOS dmg with homebrew perl.

sudo brew install perl

Then unpack the ChordPro github stuff, go into that folder, and type

sudo cpanm --installdeps .

If you then go to the folder pp/macos, you should be able to run a 'make'.

Desbeers commented 4 months ago

Unfortunately, that still does not work... I can build from Git, and with cpan but the ‘make’ in pp/macos still complains about missing ‘wxWidgets’ even when I only do a make chordpro.

I tried to install ‘wxWidgets’ but again errors...

I’m well known with homebrew and make but not too much about perl.

Anyway, I gave that a rest for the moment and updated my wrapper to a very useful state.

Main thing missing is the selection of custom files to use but I know my way around to do that. I’m well known with the sandbox restrictions and how to deal with them.

Code-signing is a different issue. The current wrapper is not signed so the familiar “the developer could not be verified” pops up. Ok, I thought that was no big deal. I compile a package, strip my useless free singing (that only works on my mac) and call it the day. Unfortunately, I found out that arm-binaries for the Mac must be signed wit a valid and paid developer account, else it will simply not run.

So, there are only three options:

So, lets give it a thought and meanwhile it would be nice if we can find another Mac user in the community who like to try my new wrapper. It looks and works very well for me!

Desbeers commented 4 months ago
Screenshot 2024-05-28 at 15 32 16
sciurius commented 4 months ago

I can build from Git,

Just to make sure: are you using the dev branch?

and with cpan but the ‘make’ in pp/macos still complains about missing ‘wxWidgets’ even when I only do a make chordpro.

In the makefile there is TARGET := wxchordpro. I think you must change that to TARGET := chordpro.

sciurius commented 4 months ago

Unfortunately, I found out that arm-binaries for the Mac must be signed wit a valid and paid developer account, else it will simply not run.

Does that apply to dmg as well?

sciurius commented 4 months ago

Let's stay optimistic and get technical. If you take a look at pp/linux/ppl.c you'll see that it is a rather trival program that sets up a couple of things and then calls the main entry point in libperl.so. Do you think that could be done directly from your SwiftUI program?

Desbeers commented 4 months ago

Let’s answer your pervious questions first.

Just to make sure: are you using the dev branch?

Yes.

In the makefile there is TARGET := wxchordpro. I think you must change that to TARGET := chordpro.

It makes no difference.

Does that apply to dmg as well?

Yes. It is just a nice packaging of the application. Same rules apply. The application in the dmg must be code signed for arm Macs.

Now lets get technical as well:

I don’t understand your last suggestion. I don’t care how a chordpro binary is created (in a technical way). All my SwiftUI wrapper is doing is calling the binary in a Swift Process() Task together with its arguments. It is very simple.

I just copied the binary, together with its requirements, from the current package into a folder inside my source to run in a Task. Reason for that is sandboxing. I can run it from a homebrew installed chordpro in exactly the same way. Just call it from a different location. That cannot be sandboxed (it is outside the package) and needs additional installations from the user.

I wrote a ‘Book Creation App’ in exactly the same way that is using Pandoc and a full Latex install. All must be installed independent with homebrew or another package manager on macOS. It is calling a very smart make to create the books so I know how to write it.

Good enough for my home projects but not for chordpro.

The self-contained binary must be part of the package. And that is were the bigger problem is right now.

Once a binary can be created, I can write an new make that builds the chordpro binary, move it into the SwiftUI wrapper ‘Bin’ directory source code (that the application itself does not care about) and create the Mac App with xcodebuild. That is part of my knowledge ;-)

We are getting close!

This is much more challenging than I thought but does not discourage me!

The SwiftUI wrapper itself is a non-issue for me, wrote it in a couple of hours with joy and any additional options are simple to implement. The foundation is in place. Settings will be preserved, build-in templates and notations are not hard-coded but directly from the source so the wrapper is very stable.

sciurius commented 4 months ago

I've checked in new version of the pp/macos/Makefile. This should be able to build cli-only with TARGET=chordpro. Can you give it a try?

sciurius commented 4 months ago

The application in the dmg must be code signed for arm Macs.

So unless we find someone to build/sign this will be the end of ChordPro for MacOS?

I've been told that intel binaries will still run on arm Macs (but you'll get a warning).

The best option would be to form an MacOS Special Interest Group within the ChordPro community. Unfortunately, most MacOS users on the list are not responsive to support questions (you being the god-sent exception).

Desbeers commented 4 months ago

I've checked in new version of the pp/macos/Makefile. This should be able to build cli-only with TARGET=chordpro. Can you give it a try?

Closer, but still complaining about missing packages. I think that is because the building is using the system perl and not the one from homebrew even tho it is in front of my $PATH. I will investigate....

Desbeers commented 4 months ago

The application in the dmg must be code signed for arm Macs.

So unless we find someone to build/sign this will be the end of ChordPro for MacOS?

I've been told that intel binaries will still run on arm Macs (but you'll get a warning).

The best option would be to form an MacOS Special Interest Group within the ChordPro community. Unfortunately, most MacOS users on the list are not responsive to support questions (you being the god-sent exception).

Correct, intel builds will still run on arm Macs. Unsigned universal builds will not.

But, good news!

I found out how to build a intel-only version, stripped the signing, uploaded it to GitHub, created a completely fresh account on my Mac, downloaded the ZIP, 2 times right-click to open and it works!

Clumsy, but progress!

So, I have a test build available:

https://github.com/Desbeers/ChordProMac/tree/main/TestBuild

sciurius commented 4 months ago

Poor me.... scrot20240529124451

Is that something you can fix, or do I need a new MacOS?

Desbeers commented 4 months ago

Is that something you can fix, or do I need a new MacOS?

I’m afraid you need a new macOS... SwiftUI is very immature on older versions than 13.

MacOS 10.15 only supports SwiftUI version 1 and that was really horrible...

I suggest to keep the current wrapper around for older Macs and just add mine as an addition for never versions.

sciurius commented 4 months ago

No problem, I need to maintain the current Wx wrapper anyway, for Linux and Windows. It's just that I'm going to be the last person to be able to admire your work :( .

sciurius commented 4 months ago

Do you need additional modifications from my side, to make it easier for you to build?

Desbeers commented 4 months ago

No problem, I need to maintain the current Wx wrapper anyway, for Linux and Windows.

Poor Linux and Windows users. My wrapper will turn ChordPro for macOS from the Ugly Duck into the most beautiful Swan of the Family, hahaha!!

It's just that I'm going to be the last person to be able to admire your work :( .

A managed to get the minimum version down to macOS 12 (not committed yet). Slightly less beautiful but a good compromise to support at least the last three versions and fundamentally the same.

I can’t go any lower... Sorry!

Do you need additional modifications from my side, to make it easier for you to build?

Thanks, but not for now. It simply does not build the command line version on my Mac. I just keep your latest bin in my source and see how to fix that later.

My TODO list:

And then...

All doable. The fact that I have a public available app now proves that it can be done!

sciurius commented 4 months ago

It simply does not build the command line version on my Mac

Can you do a fresh checkout from git,

cd ChordPro
cpanm --installdeps .
cd pp/macos
make

and send me the complete log?

Desbeers commented 4 months ago

Last login: Wed May 29 20:15:48 on ttys003 Desbeers@UberBook chordpro-dev % cpanm --installdeps . Can't write to cpanm home '/Users/Desbeers/.cpanm': You should fix it with chown/chmod first. Desbeers@UberBook chordpro-dev % sudo cpanm --installdeps . Password: --> Working on . Configuring /Users/Desbeers/Downloads/chordpro-dev ... OK <== Installed dependencies for .. Finishing. Desbeers@UberBook chordpro-dev % cd pp/macos Desbeers@UberBook macos % make rm -f .pp.deps ~ chordpro wxchordpro rm -fr build rm -fr ChordPro.app perl wxchordpro_pp.pl > wxchordpro.pp pp --output=wxchordpro @wxchordpro.pp ../../script/wxchordpro.pl /usr/bin/pp5.34: Shared library (option --link) doesn't exist: /usr/local/lib/libpng16.16.dylib at /System/Library/Perl/Extras/5.34/PAR/Packer.pm line 1216. make: [wxchordpro] Error 2 Desbeers@UberBook macos % make chordpro pp --output=chordpro @chordpro.pp ../../script/chordpro.pl /usr/bin/pp5.34: Cannot find module PDF/API2.pm (specified with -M) make: [chordpro] Error 2 Desbeers@UberBook macos %

terevos commented 4 months ago

I get the same exact error on my Mac. (M2 Pro on Sonoma 14.4.1)

Desbeers commented 4 months ago

I get the same exact error on my Mac. (M2 Pro on Sonoma 14.4.1)

Welcome to debug!

sciurius commented 4 months ago

Het commando voor cli is make TARGET=chordpro .

terevos commented 4 months ago

That command results in the same error. I'm going to try out perlbrew and see if I have any luck with that.

Homebrew's perl should not be trying to pull cpan modules from the system, I don't think. And I made sure that I have homebrew's libpng installed. (I'm not a perl guru, though)

Desbeers commented 4 months ago

(I have not played my Telecaster for days because of this GitHub issue, however, a Telecaster will will always stay in tune and she will forgive me)

sciurius commented 4 months ago

My Strat agrees :).

sciurius commented 4 months ago

That command results in the same error.

Show me. You're sure that you have the latest dev branch from git?

terevos commented 4 months ago

Ok, so I was on main before. I switched to dev and got this:

~ [16:40:01] [~/git/chordpro] [dev *]
-> % sudo cpanm --installdeps .
--> Working on .
Configuring /Users/droca/git/chordpro ... OK
<== Installed dependencies for .. Finishing.

~ [16:40:04] [~/git/chordpro] [dev *]
-> % cd pp/macos

~ [16:40:09] [~/git/chordpro/pp/macos] [dev *]
-> % make TARGET=chordpro
rm -f *.pp.deps *~ chordpro wxchordpro
rm -fr build
rm -fr ChordPro.app
pp --output=chordpro @chordpro.pp ../../script/chordpro.pl
/usr/bin/pp5.34: Cannot find module PDF/API2.pm (specified with -M)
make: *** [chordpro] Error 2

~ [16:40:16] [~/git/chordpro/pp/macos] [dev *]

The problem I see here is that it's using /usr/bin/pp5.34, which is a system perl, not homebrew. So cpanm above was using homebrew and /usr/bin/pp5.34 isn't going to use any of those modules. We either need to get the system perl to install all the appropriate cpan modules OR get the script to use homebrew's perl.

Desbeers commented 4 months ago

My Strat agrees :).

Your wishes ;-)

Dutch:

Of zo iets. Probeerde iets leuks te zeggen over Tele vs Strat.

sciurius commented 4 months ago

I work exclusively with homebrew, mostly because of wxWidgets. I think chordpro cli should be buildable with system perl. Can you install your own modules with system perl? In other words,

cd chordpro
sudo /usr/bin/cpan App::cpanminus
sudo /usr/bin/cpanm --installdeps .
Desbeers commented 4 months ago

I work exclusively with homebrew, mostly because of wxWidgets. I think chordpro cli should be buildable with system perl. Can you install your own modules with system perl? In other words,

cd chordpro sudo /usr/bin/cpan App::cpanminus sudo /usr/bin/cpanm --installdeps .

Welcome in the world of macOS where we still have Unix under the hood but with every new version its implementation is more strict. It makes sense, it is not an evil act but the reason macOS is such a stable and reliable platform.

This does not work anymore. You cannot sudo your stuff al over the place.

I have a Linux background so I know what I’m talking about.

Please give it some rest, play your Strat en let me do my thing.

Desbeers commented 4 months ago

The problem I see here is that it's using /usr/bin/pp5.34, which is a system perl, not homebrew. So cpanm above was using homebrew and /usr/bin/pp5.34 isn't going to use any of those modules. We either need to get the system perl to install all the appropriate cpan modules OR get the script to use homebrew's perl.

Yes, thats the problem. We will find a solution.

Desbeers commented 4 months ago

I’ve added a very simple Makefile to my project to build and archive the application for testing.

It’s building an unsigned, Intel only application with the usual restrictions (so 2 x right-click to open).

It requires macOS 12 or newer and runs on arm-macs as well.

sciurius commented 4 months ago

Good work.

So the release procedure becomes:

Desbeers commented 4 months ago

As for now, yes.

Still trying to get the command-line build done myself. Getting closer but stuck on HarfBuzz::Shaper now... And I found out its yours!

Desbeers@UberBook macos % make chordpro                       
pp --output=chordpro @chordpro.pp ../../script/chordpro.pl
/usr/bin/pp5.34: Cannot find module HarfBuzz/Shaper.pm (specified with -M)
make: *** [chordpro] Error 2
Desbeers@UberBook macos % sudo cpanm HarfBuzz::Shaper --force
Password:
--> Working on HarfBuzz::Shaper
Fetching http://www.cpan.org/authors/id/J/JV/JV/HarfBuzz-Shaper-0.027.tar.gz ... OK
Configuring HarfBuzz-Shaper-0.027 ... OK
Building and testing HarfBuzz-Shaper-0.027 ... FAIL
! Installing HarfBuzz::Shaper failed. See /Users/Desbeers/.cpanm/work/1717094713.18614/build.log for details. Retry with --force to force install it.
Desbeers@UberBook macos %

Any ideas?

build.log

terevos commented 4 months ago

I ran into that issue. There's something wrong with 0.027 and MacOS. Try this:

cpanm @.***

@.***

On May 30, 2024, at 2:51 PM, Nick Berendsen @.***> wrote:

As for now, yes.

Still trying to get the command-line build done myself. Getting closer but stuck on HarfBuzz::Shaper now... And I found out its yours!

@. macos % make chordpro
pp --output=chordpro @chordpro.pp ../../script/chordpro.pl /usr/bin/pp5.34: Cannot find module HarfBuzz/Shaper.pm (specified with -M) make: [chordpro] Error 2 @. macos % sudo cpanm HarfBuzz::Shaper --force Password: --> Working on HarfBuzz::Shaper Fetching http://www.cpan.org/authors/id/J/JV/JV/HarfBuzz-Shaper-0.027.tar.gz ... OK Configuring HarfBuzz-Shaper-0.027 ... OK Building and testing HarfBuzz-Shaper-0.027 ... FAIL ! Installing HarfBuzz::Shaper failed. See /Users/Desbeers/.cpanm/work/1717094713.18614/build.log for details. Retry with --force to force install it. @. macos % Any ideas?

build.log https://github.com/ChordPro/chordpro/files/15504541/build.log — Reply to this email directly, view it on GitHub https://github.com/ChordPro/chordpro/issues/373#issuecomment-2140670336, or unsubscribe https://github.com/notifications/unsubscribe-auth/AB7VSEA45XCYZH7DDA7K33DZE5YJTAVCNFSM6AAAAABIJDMJHKVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCNBQGY3TAMZTGY. You are receiving this because you are subscribed to this thread.

sciurius commented 4 months ago

This is that part of the log from the build on my system (with system perl):

jv@macky ~ % cpan HarfBuzz::Shaper
Reading '/Users/jv/.cpan/Metadata'
  Database was generated on Fri, 01 Dec 2023 17:17:02 GMT
Running install for module 'HarfBuzz::Shaper'
Running make for J/JV/JV/HarfBuzz-Shaper-0.026.tar.gz
Fetching with LWP:
http://www.cpan.org/authors/id/J/JV/JV/HarfBuzz-Shaper-0.026.tar.gz
Fetching with LWP:
http://www.cpan.org/authors/id/J/JV/JV/CHECKSUMS
Checksum for /Users/jv/.cpan/sources/authors/id/J/JV/JV/HarfBuzz-Shaper-0.026.tar.gz ok

  CPAN.pm: Building J/JV/JV/HarfBuzz-Shaper-0.026.tar.gz

NOTICE !! NOTICE !! NOTICE !! NOTICE !! NOTICE !!

We could not find a suitable harfbuzz library so we will try to
build our own. This will require a decent C++ compiler (C++11).

If building fails, see document INSTALL for (other options.

Checking if your kit is complete...
Looks good
Writing Makefile for HarfBuzz::Shaper
Writing MYMETA.yml and MYMETA.json
cp lib/HarfBuzz/Shaper.pm blib/lib/HarfBuzz/Shaper.pm
c++ -c -std=c++11 -o harfbuzz.o -DHB_NO_MT  hb_src/harfbuzz/harfbuzz.cc
/usr/bin/perl /System/Library/Perl/5.18/ExtUtils/xsubpp  -typemap /System/Library/Perl/5.18/ExtUtils/typemap -typemap typemap  Shaper.xs > Shaper.xsc && mv Shaper.xsc Shaper.c
cc -c  -Ihb_src -g -pipe -fno-common -DPERL_DARWIN -fno-strict-aliasing -fstack-protector -Os   -DVERSION=\"0.026\" -DXS_VERSION=\"0.026\"  -iwithsysroot "/System/Library/Perl/5.18/darwin-thread-multi-2level/CORE"   Shaper.c
Running Mkbootstrap for HarfBuzz::Shaper ()
chmod 644 Shaper.bs
rm -f blib/arch/auto/HarfBuzz/Shaper/Shaper.bundle
LD_RUN_PATH="/usr/lib" cc  -bundle -undefined dynamic_lookup -fstack-protector Shaper.o  -o blib/arch/auto/HarfBuzz/Shaper/Shaper.bundle harfbuzz.o   \
           -lc++        \

chmod 755 blib/arch/auto/HarfBuzz/Shaper/Shaper.bundle
cp Shaper.bs blib/arch/auto/HarfBuzz/Shaper/Shaper.bs
chmod 644 blib/arch/auto/HarfBuzz/Shaper/Shaper.bs
Manifying blib/man3/HarfBuzz::Shaper.3pm
  JV/HarfBuzz-Shaper-0.026.tar.gz
  /usr/bin/make -- OK
'YAML' not installed, will not store persistent state
Running make test
PERL_DL_NONLAZY=1 /usr/bin/perl "-MExtUtils::Command::MM" "-e" "test_harness(0, 'blib/lib', 'blib/arch')" t/*.t
t/00_load.t ..... 1/1 # Testing HarfBuzz::Shaper 0.026, harfbuzz 2.6.4i, Perl 5.018004, /usr/bin/perl
t/00_load.t ..... ok   
t/10_basic.t .... ok   
... etc ...
sciurius commented 4 months ago

0.027 uses a newer version of the harfbuzz library, please try 

cpanm J/JV/JV/HarfBuzz-Shaper-0.026.tar.gz
sciurius commented 4 months ago

FWIW, 0.027 builds equally well on my system.

sciurius commented 4 months ago

According to matrix.cpantesters.org there are several successful build for 0.027 on MacOS.

http://matrix.cpantesters.org/?dist=HarfBuzz::Shaper

Desbeers commented 4 months ago

0.027 uses a newer version of the harfbuzz library, please try

cpanm J/JV/JV/HarfBuzz-Shaper-0.026.tar.gz

That works!

And... after rebuilding PAR::Packer I did a make chordpro in the pp/macos directory....

And... IT BUILD THE BINARY!!!!! WHOHOOO!!!!!

(using system perl)

However, it is not the same as I took from the official package. It is a big beast.... But that is for another day...

sciurius commented 4 months ago

WHOHOOO!!!!! indeed!

When make TARGET=chordpro completes there are 3 chordpro binaries.

scrot20240531082344

The first (the big one) is the result from PAR packaging. It is a more or less self contained binary including all libraries and modules. It is the result from make chordpro.

The small ones are identical, specially prepared binaries linked against the perl shared library that is included in the build. The ChordPro.app filetree 'is' the chordpro app.

Desbeers commented 4 months ago

Ah! I only got a big fat ‘chordpro’ when doing make chordpro, however, make TARGET=chordpro did the trick!

I found out that make ppl TARGET=chordpro gives me enough. Then I have the ‘build’ directory with the stuff I need to stuff in my wrapper.

It didn’t work at first, found out I have to resign the ‘libperl.dylib’ with codesign.

Updated the wrapper with my self-build binary and it works!

Just cloned the *chordpro source and see if I can add my stuff with a new make...

sciurius commented 4 months ago

That sounds great!

Desbeers commented 4 months ago

You have a PR :-)