[Bug] Update build GRASS as macOS application

[nilason]

Describe the bug Building GRASS as a macOS application needs to be updated for Python 3 and long-gone packagemaker needs to be replaced.

To Reproduce Steps to reproduce the behavior:

Compiling with --enable-macosx-app using Python 3 fails.

System description (please complete the following information):

• Operating System: macOS 14+
• GRASS GIS version 7.8.2+
• Python 3 with wxPython 4

[cmbarton]

Agreed. I've been using Apple Disk Utility now for over a year. AFAICT, other packaging approaches require Xcode. Also, we need to be able to sign the code to make it easier for a binary to be opened. OSGEO has signing authority, but I have not been able to figure out how to apply it. Online descriptions of how to do this without moving the entire build to Xcode are pretty opaque and no one on the GRASS team (or OSGEO) responded to my repeated requests for advice last year. Perhaps you have suggestions about how to move forward.

[wenzeslaus]

I don't want to discuss advantages of one over another here, but I think it is important to mention here the packaging efforts for Homebrew and conda. Here is PR for recipe for conda which fails:

https://github.com/csdms-stack/grass-recipe/pull/1

And here is installation of Homebrew formula which fails too:

https://github.com/GRASS-GIS/grass-gis-experimental-ci/tree/homebrew-osgeo4mac

I discuss this a little more here: [GRASS-dev] Automatic/CI compilation on macOS

[nilason]

If I'm not mistaken, QGIS uses Homebrew for the packaging of the mac app. I could imagine taking a similar approach. It would be really good, if we in the end could create a more or less automatic build and packaging workflow (with code signing). Personally, I have no experience with Homebrew (as I use MacPorts), but that can of course change...

An important step towards this is fixing the build of a working GRASS.app (with --enable-macosx-app).

[alazarolop]

Thanks @neteler for linking the threads.

At this moment GRASS is fully working in macOS using OSGeo4Mac tap for Homebrew. The binaries are made available to the community, but one can also locally built from source based on the same formula. However, it’s not a standalone app.

AFAIK as for now QGIS app have been based on Homebrew, although it’s probably going to change due to dependencies issues. @PeterPetrik is doing a great effort in this way and could probably be a reference for it (https://github.com/qgis/QGIS-Mac-Packager).

[cmbarton]

I've been building this with Anaconda. This has largely been successful until recently. I'm using it to build self-contained apps, which most Mac users expect to have. However, this could also be transformed into an Anaconda recipe for those who want to compile themselves. Eric Hutton of CSDMS developed an initial recipe for early Python 2 versions that could be updated.

[wenzeslaus]

Just to be clear, neither conda nor Homebrew work for me in the macOS virtual machine on GitHub Actions (tested also on Travis in case of Homebrew) and I don't know what to do next (latest email from Rainer M Krug on grass-dev). Please, let me know if you have any suggestions (grass-dev, issues, PR).

Homebrew install test (fork here):

https://github.com/GRASS-GIS/grass-gis-experimental-ci/runs/487969965?check_suite_focus=true

conda recipe test (based on Eric Hutton's work, fork updated version here):

https://github.com/wenzeslaus/grass-recipe/runs/486679988?check_suite_focus=true

[alazarolop]

@wenzeslaus you are right, I can reproduce it with the code from the email and the latest Homebrew release (with Proj 7.0.0). This should be a new issue though, since the version using Proj 5.2.0 was running fine except for the GUI (not sure about Proj 6.3.1 that was released in february).

Oddly, now just executing grass78 let you run GRASS both from command line and GUI.

[neteler]

AFAIK, GRASS GIS does not yet support PROJ 7.x, see issue #395

[nilason]

@wenzeslaus

conda recipe test (based on Eric Hutton's work, fork updated version here):

https://github.com/wenzeslaus/grass-recipe/runs/486679988?check_suite_focus=true

According to build log (line 120) python 3.7.6 is used. If this is the case, it is doomed to fail because of changes in that version for ctypes. 3.7.7 should work (or 3.8.2), with latest GRASS 7.8 git-branch version (https://github.com/OSGeo/grass/commit/7a9ce5d5ced8c9409464908171759865239abbe2 is a critical fix) or master.

[nilason]

FYI, compiling and running on mac (10.14) from terminal is working fine for me with following dependencies:

• Python=3.7.7 (official release)
• wxPython 4.0.7.post2 osx-cocoa (phoenix) wxWidgets 3.0.5 (installed from pip3)
• proj=6.3.1
• gdal=3.0.3
• geos=3.8.1
• sqlite=3.31.1
• poppler=0.87

[cmbarton]

Thanks for this. I was able to compile 7.9 last Friday by downgrading to Python 3.7.4. If I can get it compile consistently, I will happily share my protocol for building a Mac app.

The digitizer works for me at last!! Thanks so much to you and Anna for this and other fixes. I will try to also compile 7.8 dev today.

[cmbarton]

One annoying problem I am having I could use some advice from anyone on is about Git. I am not writing or modifying code at the moment, just trying to catch up compiling binaries. I would like to regularly update my local repository from the main GRASS site but do not want to push anything out. I do need to make changes to a few files before I compile (Eric Hutton's patches a bit modified), but at the moment want to keep them entirely local.

But I can't seem to find anything in Git that works like the old svn up. Ideally, I'd like to update everything except my patches. But if I can't do that, I have no problem redoing the patches before I compile.

I've tried to fork my own version (with fetch upstream) and I've tried just to update my repository directly from the osgeo.org repository. I've tried rebase with a stash and even reset --hard. I simply cannot get Git to update if I've made any changes locally.

Any suggestions are welcome. Thanks.

[neteler]

@cmbarton try this:

git clone curl https://github.com/OSGeo/grass.git
cd grass
git pull

Subsequent updates simply with git pull.

[cmbarton]

Thanks. I'll try that.

Michael

On Mar 31, 2020, at 10:35 AM, Markus Neteler notifications@github.com wrote:

@cmbarton https://github.com/cmbarton try this:

git clone curl https://github.com/OSGeo/grass.git cd grass git pull Subsequent updates simply with git pull.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/OSGeo/grass/issues/457#issuecomment-606769850, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACENLLY2KJ4OC3CUNBONQDDRKISUJANCNFSM4LSZE6AA.

[PeterPetrik]

Hi, it looks like that if LIB environment variable is set, the compilation or gpde fails with "missing symbols __G_calloc" and others. From the link command it looks like the libraries like gis and gmath are simply not added to the compile line. for complitation I had to do this fix https://github.com/qgis/QGIS-Mac-Packager/blob/f12e73d0867c49a9f941127ab902eb7eaf5362c7/qgis_deps/recipes/grass/recipe.sh#L70

[cmbarton]

I think and hope that I may now finally have a consistently reproducible procedure for building self-contained apps again using condo. Here it is in brief schematic. I'm happy to elaborate if anyone is interested in the details.

1. Create an app shell for the GRASS version (easiest to modify one that Eric Hutton first made)
2. Install a minimum anaconda distribution (miniconda) into the shell and then GRASS dependencies, including Python
3. Set the path to the app shell
4. Do a regular configure, build and install, pointing prefix into the app shell
5. Put the app shell with GRASS, dependencies, and python into a folder with a symlink to the Applications folder
6. Use disk utility to create a binary package for distribution

Currently Anaconda is defaulting to Python 3.7.6, which is the only one that does not currently work with building and running GRASS. While I can create environments for different versions, the miniconda installer defaults to 3.7.6 when installing into the GRASS shell. I've tried various other version in my requirements file (for dependencies), but right now, the only Python version that installs, builds, and runs consistently is 3.6.10. So I'm using that.

[wenzeslaus]

@cmbarton, I don't get past first point on macOS on GitHub Actions [code, workflows]. Suggestions, PRs welcome. If there are is a list of dependencies or local changes you are using, these need to go to OSGeo/grass or another repo.

Unfortunately, the versions suggest by @nilason don't work for conda even when I loosen the requirements a bit. I'm getting a lot of decency conflicts basically for all the dependencies. (I got the dyld: Library not loaded: @rpath/libpoppler.76.dylib error before that.) So, different combo of dependencies is needed for conda.

@nilason, if you want to add MacPorts-based build to https://github.com/GRASS-GIS/grass-gis-experimental-ci/ that would be great. The idea is that once these are work reasonably well, we add them to this repo (OSGeo/grass), so we have the build testing in place. (Speaking about those which can build from the local source code which now not done in GRASS-GIS/grass-gis-experimental-ci since it is another repo and also because existing recipes/packages are used such as the one from homebrew-osgeo4mac.)

[cmbarton]

I’ll put together a detailed set of procedures soon for all

[cmbarton]

grassmac_supporting_files.zip configure.sh.zip configure.sh.zip

Using Anaconda to Create a Binary Mac App for GRASS

These instructions are for creating a self-contained Mac app, including all dependencies, with GRASS 7.9 and Anaconda. You can modify them if you don't need some of the steps. Some of the steps you won't need to repeat (e.g., creating an anaconda environment)

1. Create a Mac app shell (I am uploading an example, modified from shells created by Eric Hutton).

This can be modified serve for different versions of GRASS by altering the last line of the script that launches GRASS in the app shell. For example, modify the last line of ...

../GRASS-7.9.app/Contents/MacOS/Grass.sh "$GRASS_PYTHON" "$GISBASE/bin/grass79" "-gui" "$@"

to refer to grass78 or grass76

2. Create an anaconda environment for Python 3.6.10

This may not be strictly necessary, but I am doing it make sure that my builds use Python 3 instead of Python 2.7, which is still what ships with the Mac.

Anaconda defaults to Python 3.7.6, which will not compile GRASS properly. This makes it more complicated than needed, requiring some workarounds. I've tried this with multiple versions of Python 3 (anaconda has them all the way up to 3.8.2), but the highest version that seems to give consistently successful results is 3.6.10

So I create an anaconda environment for Python 3.6.10

conda create anaconda_p36 python==3.6.10 anaconda

3. Activate the anaconda Python 3.6.10 environment

conda activate anaconda_p36

4. Copy the Mac app shell to /Applications

cp -r ../GRASS-7.9.app /Applications/GRASS-7.9.app  

5. Download the Python 3 miniconda installer and use it to install a minimal anaconda environment into the Mac app shell

curl https://repo.continuum.io/miniconda/Miniconda3-latest-MacOSX-x86_64.sh > miniconda3.sh 

bash ./miniconda3.sh -b -f -p /Applications/GRASS-7.9.app/Contents/Resources    

6. Export the path to the GRASS Mac app shell so that subsequent operations use the environment of that shell

export PATH=/Applications/GRASS-7.9.app/Contents/Resources/bin:$PATH   

7. Install GRASS dependencies into Mac app shell, including Python 3.6.10

This will downgrade Python from v. 3.7.6 installed by miniconda to the more useable 3.6.10.

I use a requirements file for all the dependencies and will upload it along with the app shell

conda install --yes -p "/Applications/GRASS-7.9.app/Contents/Resources" --file=requirements_wxp4p36.txt -c conda-forge  

8. Create a symlink to Python so that pythonw can find it

Grass launches Python through the pythonw script. But the script does not point to the correct location for Python (which is in fact in 2 different places). This seems to be a bug in Python < 3.7.4 or how anaconda installs it. It puts the python launch file in the wrong place. So a symlink is needed. Hopefully this will go away when we can use Python > 3.7.6, which seems to put python in the correct place, and the place that pythonw points to.

ln -sf /Applications/GRASS-7.8.app/Contents/Resources/python.app/pythonapp/Contents/* /Applications/GRASS-7.8.app/Contents/Resources/python.app/Contents

9. cd to the ./grass source folder and apply patches to configure and make files

Eric Hutton worked out these patches and I modified one needed to use Python 3. Maybe they can be worked into the source code so that they are no longer needed. I will upload the patches along with the app shell and requirements file.

patch -p0 < aclocal.m4.patch  
patch -p0 < configure.patch  
patch -p0 < install.make.patch  
patch -p0 < loader.py.patch  
patch -p0 < module.make.patch  
patch -p0 < platform.make.in.patch  
patch -p0 < rules.make.patch  
patch -p0 < shlib.make.patch

10. Configure, make, and install

I use a configure script that sets flags and prefix. I will include this in my package of uploads

bash ../configure.sh
make -j4  GDAL_DYNAMIC=
make -j4 install  

You should now be able to run and test the GRASS-7.9.app in your /Applications folder

11. Clean up

This removes unneeded anaconda package installer files and some other large files that are irrelevant for GRASS

conda clean --all  -y  
rm -r /Applications/GRASS-7.9.app/Contents/Resources/mkspecs  
rm -r /Applications/GRASS-7.9.app/Contents/Resources/phrasebooks  
rm -r /Applications/GRASS-7.9.app/Contents/Resources/pkgs  
rm -r /Applications/GRASS-7.9.app/Contents/Resources/plugins  
rm -r /Applications/GRASS-7.9.app/Contents/Resources/qml  
rm -r /Applications/GRASS-7.9.app/Contents/Resources/resources  
rm -r /Applications/GRASS-7.9.app/Contents/Resources/translations 
rm /Applications/GRASS-7.9.app/Contents/Resources/lib/libQt*  

12. Set up folder to make a binary package for distribution

Put a copy of the new GRASS app and a symlink to the /Applications folder into a new folder that will become the distribution package

mkdir ../binary_pkg/grass-7.9
cd ../binary_pkg/grass-7.9
cp -R /Applications/GRASS-7.9.app ./
ln -s /Applications Applications  

13. Use Disk Utility to make a distribution package

• Open disk Utility
• From the make file/new image menu, select image from from folder
• Point it to the folder with the app and symlink to /Applications and tell it to make an image

grassmac_supporting_files.zip

[PeterPetrik]

@wenzeslaus I am about to finish using workflows for macos CI builds for QGIS, see https://github.com/qgis/QGIS/pull/35407/files#diff-e58291fad8cbc1e6be9085e97679afdd (probably you can use version qgis-deps-0.2.1 (will be uploaded early next week) of the qgis-deps package to do similar for grass).

[cmbarton]

I forgot to include my configure script last night. I just now tried to add it to my zip file or separately. For some reason, I could not add it to my comment. So it is here.

configure.sh.zip

[wenzeslaus]

@cmbarton I used your version requirements and I get these two errors (same as before):

# in configure step:
found PROJ version 6.2.1
using new PROJ version 5+ API
checking for location of External PROJ library... $PREFIX/lib
checking for proj_pj_info in -lproj... yes
checking for location of External PROJ data files... $PREFIX/share/proj
checking for $PREFIX/share/proj/epsg... no
configure: `warning: *** Unable to locate PROJ data files.`
...
# throughout the code base:
dyld: Library not loaded: @rpath/libpoppler.76.dylib
  Referenced from: $PREFIX/lib/libgdal.26.dylib
  Reason: image not found

The only differences I see, besides shorter list of dependencies, is missing --with-macosx-sdk=, --with-macosx-archs="x86_64" and couple dependencies (such as zstd) in configure. I don't see a connection between the error messages and the differences I see and I don't want to try things randomly.

[cmbarton]

I've messed with both of these in several ways and I'm not sure of the impacts. But I sometimes have trouble if I don't include them. So I do. I've all the SKDs back for several years. Originally, I was building against 10.9 or so to try and keep some backward compatibility (though I don't know if it matters or not). Because of all of the difficulty in compiling in the past 9 months, I've moved to just using the current one that comes with Xcode. I guess I'd put it in. The macosx-archs argument was originally to allow for dual 32/64 bit code. With the move to wxPython >2, I was able to drop that and just do 64 bit. Again, I am not sure it is needed or not.

As to the proj/gdal error, it looks like it is not looking in the right place for proj.

Did you

1. make sure to install all dependencies, including gdal and proj, via conda into the Mac app shell?
2. And did you make sure to change your path so it will look at GRASS-7.9.app/Contents/Resources/bin first?

The configure script I use sets prefix to match what Python sees when pathed to the app. What does echo $PREFIX show?

[wenzeslaus]

Thank you @cmbarton for suggestions. I used --with-macosx-sdk set to $CONDA_BUILD_SYSROOT which resolves to

/Applications/Xcode_11.3.1.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk

It still fails with Library not loaded: @rpath/libpoppler.76.dylib.

Did you

1. make sure to install all dependencies, including gdal and proj, via conda into the Mac app shell?

I don't know what you mean by Mac app shell, but I specify my dependencies in meta.yaml and they are installed in the by conda build.

And did you make sure to change your path so it will look at GRASS-7.9.app/Contents/Resources/bin first?

I'm not touching any paths. That's what conda is doing. So, I don't know how GRASS-7.9.app/Contents/Resources/bin plays into that if it has any meaning in conda at all. However, PATH seems to have the $PREFIX as a first entry:

export PATH=/usr/local/miniconda/conda-bld/grass_1585944550702/_h_env_placehold_placehold_placehold.../bin:/usr/bin:/bin:/usr/sbin:/etc:/usr/lib

The configure script I use sets prefix to match what Python sees when pathed to the app. What does echo $PREFIX show?

Prefix is set by conda build like this:

export PREFIX=/usr/local/miniconda/conda-bld/grass_1585924255168/_h_env_placehold_placehold_placehold...

Since that would be really inconvenient to see in the outputs, conda replaced that by $PREFIX, so that's what you see in the log except for the export line.

[cmbarton]

OK. This is a different approach than what I'm doing. What I do is not directly translatable into building this within Anaconda. What might work is the following (maybe you are doing this).

1. Create a new conda environment with Python 3.6.10, e.g.,

   conda create -n grass79_conda python==3.6.10 anaconda

2. Start the new environment

   conda activate grass79_conda

3. Use my requirements file to install all dependencies into that environment (you won't need Python)
4. While still in that environment, cd to ./grass, configure and build, setting prefix to ../anaconda/envs/grass79_conda (wherever you put your anaconda distribution)
5. Install

This should create a version of GRASS that launches like the Linux one, but does so in the conda environment you created. You will have to activate that environment to run GRASS. e.g.,

conda activate grass79_conda
grass79

[wenzeslaus]

That's pretty much what conda build is doing as far as I can tell. It creates environment, actives it and builds GRASS inside.

From your conda create command, it seems you are installing anaconda package. What does it do? (It is little difficult to search for it; even for whether it even exists.)

Related to that, one question which comes to my mind is what channels you are using. I'm using conda-forge and csdms-stack because the Eric's grass-recipe was for csdms-stack:

conda config --add channels conda-forge
conda config --add channels csdms-stack

[wenzeslaus]

@PeterPetrik, thanks I'm excited to see this. Please let me know when that's finished. If you have any suggestions how to adapt it for building GRASS in CI, please let me know (build of a standalone app, conda package, Homebrew formula are desired, but any build will do for starters for CI). I looked at the qgis/QGIS-Mac-Packager repo and it was not clear to me how to easily reuse it (without actually using the whole thing), but .github/workflows/macos-build.yml looks like a better starting point, but it still seems too much, so any tips would be appreciated (like on what to safely ignore).

[PeterPetrik]

@wenzeslaus

@PeterPetrik Please let me know when that's finished. If you have any suggestions how to adapt it for building GRASS in CI, please let me know.

Can you please write me to my email peter.petrik@lutraconsulting.co.uk to discuss the topic, maybe even in the call?

[cmbarton]

@wenzeslaus @PeterPetrik Replying to your query from last night.

Environments

Anaconda environments are a sort of virtual machine. You can create a specific configuration of anaconda installation and packages. As I suggested, even though Anaconda defaults to Python 2.7 on the current Mac, you can create environments with different versions of Python and other packages.

What I was suggesting was to make an Anaconda environment specific to GRASS. Then it could have just the packages and versions it needs to run GRASS. No need to change the main Anaconda installation.

To keep it minimal and light, you could do the following

conda create --name grass79_conda
activate grass79_conda
bash ./miniconda.sh # run the miniconda installer
conda install python==3.6.10
conda install --yes --file=requirements_wxp4p36.txt -c conda-forge   
conda clean --all  --yes

After this, you could configure, build, and install GRASS into the environment

bash ../configure.sh
make -j4  GDAL_DYNAMIC=
make -j4 install  

To exit the environment...

conda deactivate

Channels

I am only using the main conda channel and conda-forge. I am not using the csdms channel because I don't think Eric has updated it recently and don't want to inadvertently get unworkable versions of main packages or GRASS dependencies.

[wenzeslaus]

Thanks @cmbarton, GDAL_DYNAMIC= looks like something which could be related to the problem I'm having. It also seems that now I have enough info to transfer your compilation steps into a workflow for GitHub Actions, so I will get back to you once I'm done.

@PeterPetrik, email sent. Thanks.

[cmbarton]

That's great @wenzeslaus. Let me know if I can help fill in any other details.

[wenzeslaus]

I reproduced the configuration for a build according to @cmbarton as a workflow for GitHub Actions. There are some minor changes such as dependencies being in environment.yml file instead of requirements.txt due to how conda works in a non-interactive/script environment and with GitHub Actions. Let me know if you see any differences. The code is in conda-compile branch of GRASS-GIS/grass-gis-experimental-ci. You can open issues or PR against that repo if you want.

The build fails in the ./configure with C compiler cannot create executables...libSystem.tbd...file was built for unsupported file format...which is not the architecture being linked (x86_64) based on the log:

2020-04-07T13:50:40.2858300Z checking host system type... x86_64-apple-darwin19.4.0
2020-04-07T13:50:40.2860260Z checking for gcc... x86_64-apple-darwin13.4.0-clang
2020-04-07T13:50:41.2879760Z configure: error: installation or configuration problem: C compiler cannot create executables.
2020-04-07T13:50:41.2881320Z checking whether the C compiler (x86_64-apple-darwin13.4.0-clang -march=core2 -mtune=haswell -mssse3 -ftree-vectorize -fPIC -fPIE -fstack-protector-strong -O2 -pipe -Wl,-pie -Wl,-headerpad_max_install_names -Wl,-dead_strip_dylibs) works... no
...
2020-04-07T13:50:41.3100710Z ld: warning: ignoring file /Applications/Xcode_11.3.1.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/usr/lib/libSystem.tbd, file was built for unsupported file format ( 0x2D 0x2D 0x2D 0x20 0x21 0x74 0x61 0x70 0x69 0x2D 0x74 0x62 0x64 0x2D 0x76 0x33 ) which is not the architecture being linked (x86_64): /Applications/Xcode_11.3.1.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/usr/lib/libSystem.tbd
2020-04-07T13:50:41.3100850Z ld: dynamic main executables must link with libSystem.dylib for architecture x86_64
2020-04-07T13:50:41.3101540Z clang-4.0: error: linker command failed with exit code 1 (use -v to see invocation)

I could try to random things like removing the --with-macosx-archs=x86_64 line, but I would like to get some feedback first on what I have now.

[cmbarton]

One difference is that I hard-coded a path to the SDK. That is not a solution for a generic conda build of course. But you might try that to see if it is the cause of the error.

[nilason]

I also think this is related to sdk versions. Perhaps https://docs.conda.io/projects/conda-build/en/latest/resources/compiler-tools.html#macos-sdk can throw some light on this.

[cmbarton]

I'll also copy here what I sent to the dev list. I've got a cleaned up version of instructions for building a binary app using Anaconda. Any thoughts as to where the best place to put it? Wiki? Other?

[nilason]

The initial intent (and still is) with reporting this issue is to enable --enable-macosx-app to build a mac app bundle. The comments/discussions have though taken a direction into more of packaging/dependency problematics, which is in itself not bad, but just a much complicated matter.

I'll take the moment to inform followers of this "thread" of advances in building for mac (using conda) on CI (see successful CI build on https://github.com/nilason/grass-gis-experimental-ci/tree/conda-updates). I have also started work on a script for an all-in-one solution for building from source to creating dmg file with GRASS.app (also using anaconda). It it very reliable for me, but there are surely room for improvement, see https://github.com/nilason/grass-conda.

[echoix]

Is this issue still valid, since we have been building for macOS in CI and have releases for macOS. Close if already resolved!

[nilason]

Is this issue still valid, since we have been building for macOS in CI and have releases for macOS.

It is unfortunately still valid, the CI build is a classic unix like build and not app bundle (see eg https://en.wikipedia.org/wiki/Bundle_(macOS)). The official release is created with https://github.com/nilason/grass-conda and the procedure is not incorporated in GRASS make files.

[echoix]

Thanks for the answer!

If someone wants to continue on this, but doesn't have access to a macOS to test afterwards, would you be willing to help?

The steps in your repo, in their current state, takes approximately how much time to run? Is something based on this suitable to run on CI? Is it still possible to build a Apple-Silicon binary using an intel cpu (as the ones freely available by GitHub actions provide)?

Are they other blockers we should be aware of?

[cmbarton]

I can test too.

[echoix]

I can test too.

What type of machine do you have, is it intel or M1-M3, or you have access to both ?

[nilason]

Thanks for the answer!

If someone wants to continue on this, but doesn't have access to a macOS to test afterwards, would you be willing to help?

I have planned all along to add this to GRASS repo myself, but in a more appropriate way. I still have it high on the todo list.

The steps in your repo, in their current state, takes approximately how much time to run? Is something based on this suitable to run on CI? Is it still possible to build a Apple-Silicon binary using an intel cpu (as the ones freely available by GitHub actions provide)?

Both the CI build and the bundle build are based on conda dependencies, but there is not much sense to build a bundle just for CI testing. It would however be a great improvement to create release packages automatically. For that ends GH, lacking Apple ARM runners (at least last time I checked), would not do. But I believe there are alternatives to GH for that.

Are they other blockers we should be aware of?

There is a potential issue for macOS 14 reported with https://lists.osgeo.org/pipermail/grass-dev/2023-September/096038.html, which may need some reworking on the start-up procedure.

[echoix]

I have planned all along to add this to GRASS repo myself, but in a more appropriate way. I still have it high on the todo list.

Is it ok with you if I start by including what exists already on your repo (and that builds the releases anyways), and we "build" (as on work) from that to have something better than nothing?

The steps in your repo, in their current state, takes approximately how much time to run? Is something based on this suitable to run on CI? Is it still possible to build a Apple-Silicon binary using an intel cpu (as the ones freely available by GitHub actions provide)?

Both the CI build and the bundle build are based on conda dependencies, but there is not much sense to build a bundle just for CI testing.

Since this is currently what is used to release the official release, I consider that it is a bigger risk to not follow the health of our "production" build. There's little benefit to build something that isn't used at all, unless the unused build is way faster and still uncovers problems, and allows to run the conda-based Mac build less often.

It would however be a great improvement to create release packages automatically. For that ends GH, lacking Apple ARM runners (at least last time I checked), would not do. But I believe there are alternatives to GH for that.

This fall they finally released macOS M1 runners, but they don't offer it free to all. They are part of their "larger" runners, that are pay per use. I kinda understand that they need to have a certain ROI for this, and it was probably really not profitable to acquire the hardware, and find a licensing agreement with Apple, as they usually only allow "renting" a whole system for a complete 24h. And the existing intel-based macOS runners still have a little life left in them ;) https://github.blog/2023-10-02-introducing-the-new-apple-silicon-powered-m1-macos-larger-runner-for-github-actions/

[cmbarton]

I can test too.

What type of machine do you have, is it intel or M1-M3, or you have access to both ?

M2. I might be able to get an intel if needed. But those are now several years out of date.

[cmbarton]

I have planned all along to add this to GRASS repo myself, but in a more appropriate way. I still have it high on the todo list.

Is it ok with you if I start by including what exists already on your repo (and that builds the releases anyways), and we "build" (as on work) from that to have something better than nothing?

The steps in your repo, in their current state, takes approximately how much time to run? Is something based on this suitable to run on CI? Is it still possible to build a Apple-Silicon binary using an intel cpu (as the ones freely available by GitHub actions provide)?

Both the CI build and the bundle build are based on conda dependencies, but there is not much sense to build a bundle just for CI testing.

Since this is currently what is used to release the official release, I consider that it is a bigger risk to not follow the health of our "production" build. There's little benefit to build something that isn't used at all, unless the unused build is way faster and still uncovers problems, and allows to run the conda-based Mac build less often.

It would however be a great improvement to create release packages automatically. For that ends GH, lacking Apple ARM runners (at least last time I checked), would not do. But I believe there are alternatives to GH for that.

This fall they finally released macOS M1 runners, but they don't offer it free to all. They are part of their "larger" runners, that are pay per use. I kinda understand that they need to have a certain ROI for this, and it was probably really not profitable to acquire the hardware, and find a licensing agreement with Apple, as they usually only allow "renting" a whole system for a complete 24h. And the existing intel-based macOS runners still have a little life left in them ;) https://github.blog/2023-10-02-introducing-the-new-apple-silicon-powered-m1-macos-larger-runner-for-github-actions/

Just a note. The current separate conda build (from Nicklaus) I use for my releases is very fast and pretty much error free.

[echoix]

I have a first draft started at https://github.com/echoix/grass/pull/15, to make sure everything is possible. (I added a job in the macOS workflow that simply clone the grass-conda repo for now, until I'm close enough to integrate directly the functionality).

I'm a little stuck, but didn't search for hours neither yet. Once the app is built, how do you use it in the command-line on macOS? Command grass on the command-line isn't available, so existing scripts don't work (yet) without changes. Is it GRASS.8.4.app or something like that? If so, is there a way to not have the version number inside the name, to not have to change anything at each version?

See this run, where I tried running the print versions script https://github.com/echoix/grass/actions/runs/7318245039/job/19934677740#step:7:1