search

Maproom / qmapshack

Consumer grade GIS software
GNU General Public License v3.0
297 stars 64 forks source link

QMapShack for macOS #480

Open BP22190 opened 2 years ago

BP22190 commented 2 years ago

Is your feature request related to a problem? Please describe.

Till version 1.15.2, there was macOS binary assets for QMapShack (and for Windows), plus source files. Now, it seems there is only some Linux binary distros and Windows, but no more macOS binaries.

So, how to upgrade QMapShack on a macOS hardware ?

Describe the solution you'd like

I would like to be able to continue to use QMapShack on my macBook ...

Describe alternatives you've considered

None actually, please suggest one... ;-)

Give a reason why someone else should spend time on that request

I don't understand why there was macOS binaries till version 1.15.2, and no more after : are Apple users banned ?

Additional context

Another wish... Apple migrates from Intel to ARM/M1 : so, it would be nice to have also an ARM binary distro for QMapShack... possible ?

kiozen commented 2 years ago

This is an Open Source project - no one is banned. The project itself is provided and maintained as source. All binaries on the download page are provided by volunteers spending their spare time to create binaries and to bundle them. For OS X no one is providing binaries at the moment. There is no special reason. It's just that no one cares about it.

You want a suggestion for an alternative: Build QMapShack on your own. And if you feel lucky create a bundle to be uploaded to the download page.

kiozen commented 2 years ago

This partially duplicates ticket #377

kiozen commented 2 years ago

This duplicates ticket #458

BP22190 commented 2 years ago

Build QMapShack on your own.

Not so easy... in general, the vast majority of macOS (and Windows) users do not have the ability to build software themselves. The development tools are not part of the standard distributions.

kiozen commented 2 years ago

The development tools are not part of any operating system distribution. You always have to install them. All tools and dependencies to build QMapShack are Open Source. But - I agree - you have to spend time. But what other option do you have if no one offers his/her time for free?

There is even a Howto in the repo

https://github.com/Maproom/qmapshack/blob/dev/MacOSX/HowtoBuildOSX.txt

But keep in mind it's outdated. You need to use more recent libraries for GDAL and PROJ for sure.

Anyways this is all the result of the OS X community not caring about this project. (or open source projects in general?) For the other platforms users update the documentation and provide binaries. So it's really up to those using OS X.

BP22190 commented 2 years ago

HowtoBuildOSX.txt ... But keep in mind it's outdated

Surelly... macOS is no longer called "OSX" since 2016 (macOS 10.12, Sierra)... ;-)

Best regards

kiozen commented 2 years ago

A don't care for me ;) Using Linux.

BP22190 commented 2 years ago

Using Linux.

Me too... but mainly for development. For my recreational and family use, I'm using macOS (and sometimes Windows), nobody is perfect... ;-)

BP22190 commented 2 years ago

Hello,

I'm trying to build a macOS distro, but have problems because the "howtobuildosx" is not up to date.

My main problem for the moment is this error (same kind for other packages) when executing cmake : 

CMake Error at CMakeLists.txt:158 (find_package):
  By not providing "FindPROJ.cmake" in CMAKE_MODULE_PATH this project has
  asked CMake to find a package configuration file provided by "PROJ", but
  CMake did not find one.

  Could not find a package configuration file provided by "PROJ" with any of
  the following names:

    PROJConfig.cmake
    proj-config.cmake

  Add the installation prefix of "PROJ" to CMAKE_PREFIX_PATH or set
  "PROJ_DIR" to a directory containing one of the above files.  If "PROJ"
  provides a separate development package or SDK, be sure it has been
  installed.

So, what to do... ?

Thanks in advance,

Best regards,

kiozen commented 2 years ago

How did you build Proj8? If you used cmake the compile process should have generated a PROJConfig.cmake, too. This should be in the cmake module path.

As an alternative you might use the files in ./cmake/Modules but they might not fit MacOS.

BP22190 commented 2 years ago

How did you build Proj8?

Not build, installed from MacPorts

As an alternative you might use the files in ./cmake/Modules

Hmmm... no such file PROJConfig.cmake there...

Will try to build Proj8 from scratch.

kiozen commented 2 years ago

Ah yes, sorry. The files where removed as Proj8, GDAL and Quazip provide proper cmake files now.

Another alternative would be to add the paths in the main CMakeLists.txt just as Windows does it. However I would prefer it if there is a proper solution that does it like "it has to be done properly on MacOS"

BP22190 commented 2 years ago

OK. Seems to solved for PROJ (I found a working cmake file on the Web), but now, same problem with "FindQuaZip-Qt5.cmake"... and don't find this one...

eyzdh commented 2 years ago

OK. Seems to solved for PROJ (I found a working cmake file on the Web), but now, same problem with "FindQuaZip-Qt5.cmake"... and don't find this one...

Did you found a way to have? I have on my mac just 1.15.2 and would like to install 1.16.1 but i have no knowledge to build for OS X

BP22190 commented 2 years ago

Hello eyzdh,

Unfortunately, I gave up. Too much trouble, the HowToBuildOSX note is outdated and unusable. And since I have no help here, I don't know what to do.

I could compile and create a version for macOS, but not without a little help, I don't know the architecture of the project and without that, it's very difficult to understand what to do. Every time I fix a problem, two come up. It's clear to me that it's possible to produce a macOS version if the developers of the project help, not if they consider it pointless...

Til version 1.5, there were macOS versions, and no binary versions for Linux here. Since version 1.6, it's the opposite ! This is a significant turnaround ! Why did they abandon Macs and their users ?

Best regards,

hhornburg commented 2 years ago

OK. Seems to solved for PROJ (I found a working cmake file on the Web), but now, same problem with "FindQuaZip-Qt5.cmake"... and don't find this one...

Hi, does this file help? https://github.com/Maproom/qmapshack/blob/af90f63693974119a5e92efb57a7f8cc0c13a835/FindQuaZip5.patch (It's the patch for the cmake quazip file from the repo at the tag 1.15.1)

BP22190 commented 2 years ago 
...
-- Found GDAL: /opt/local/lib/libgdal.dylib (found version "3.4.0") 
-- Could not find PROJ using internal code.
-- Found JPEG: /opt/local/lib/libjpeg.dylib (found version "80") 
-- Found ROUTINO: /opt/local/lib/libroutino.dylib
-- Found ROUTINO: /opt/local/include
-- Found ROUTINO: /opt/local/share/routino
-- Found ZLIB: /opt/local/lib/libz.dylib (found version "1.2.11") 
-- Could not find ALGLIB using internal code.
CMake Error at CMakeLists.txt:169 (if):
  if given arguments:

    "VERSION_LESS" "7.2.1"

  Unknown arguments specified

-- Configuring incomplete, errors occurred!

My PROJ and ALGLIB cmake files seems not working...

kiozen commented 2 years ago

Yes, the build environment variable PROJ_VERSION is not found. This happens because the line 

find_package(PROJ                   REQUIRED)

does not set it. Looks like your cmake script for PROJ does not really match the one that is build when compiling PROJ8 with cmake.

BP22190 commented 2 years ago

Hi,

So, now, the cmake is OK. But, when compiling (make) I got:

/Users/administrateur/Developpement/QMapShack/src/qmapshack/../common/gis/proj_x.h:22:10: fatal error: 'proj.h' file not found
#include <proj.h>
         ^~~~~~~~
1 error generated.

So, another problem with PROJ... (proj.h exist and is in right place !)

kiozen commented 2 years ago

It's hard to help if you do not provide detailed information about what you are doing. Please keep in mind those CMake find scripts are not made to satisfy cmake. They are made to provide basic information about a library package. The most basic information is

Modern scripts provide also the version of the library to catch API incompatibilities.

Therefore there is not much sense in just tinkering the cmake variables to satisfy the cmake run. You have to do it correctly to provide all the information needed with valid paths.

A long story short: PROJ_INCLUDE_DIRS does not provide the correct path to the include files of PROJ8.

Other variables that should be set correctly are 

${GDAL_INCLUDE_DIRS}
${PROJ_INCLUDE_DIRS}
${ROUTINO_INCLUDE_DIRS}
${ALGLIB_INCLUDE_DIRS}

${GDAL_LIBRARIES}
${PROJ_LIBRARIES}
${ROUTINO_LIBRARIES}
${ALGLIB_LIBRARIES}

And for the QuaZip the properties structure QuaZip::QuaZip has to be set.

BP22190 commented 2 years ago

Thanks, but...

Understand, I can help build a macOS version by following a reliable procedure, and with scripts that work. But I'm not a QMapShack developer, or even a developer at all under macOS.

I keep coming across inconsistencies: for example, I have to figure out that you have to install PROJ, but it's called PROJ4 everywhere, and in fact, it's PROJ8 that needs to be installed ! It may be obvious to you, put for me, it's nonsense!

For me "for the QuaZip the properties structure QuaZip::QuaZip has to be set" means nothing...

I ended up managing to run the cmake until the end, but then, the compilation (make) only produces executables which seem abnormally small to me (12Mb for qmapshack). And trying to make it a macOS app with provided scripts doesn't produce anything executable. I don't know how to find the problems, there seem to be too many.

I've already spent a lot of time, but I give up, I can't run a process that no longer works for macOS.

Best regards

kiozen commented 2 years ago

Surprise, surprise, some binaries. Please test before i publish them.

https://magentacloud.de/s/iPTCAaSBKpHNHa5

BP22190 commented 2 years ago

Hi,

Just tried : it works. I.e., it starts without problem, not yet an intensive test...

Best regards,

kiozen commented 2 years ago

I opened a PR with some fixes and an updated bundle script and Howto. This is not stuff made by me. I just collect it.

https://github.com/kiozen/qmapshack/tree/QMS-497

It would be really helpful to check if that enables others to build everything for MacOS. If not we need to figure out the missing parts and fix the files. So, if you want to improve the situation for MacOS your help is much appreciated.

Regarding the M1 processor: There seems to be no ready to use binary version of Qt5. Only for Qt6. Switching to Qt6 requires quite a lot of changes in the code. And it will break backward compatibility to older Qt5 versions that are required by distributions. I do not consider the classic #ifdef hell as an alternative. Therefore M1 support is only possible via Rosetta for now.

BP22190 commented 2 years ago

Hi,

OK, I will take a look. But surely not before Friday, not the time before...

Best regards,

PS : regarding M1 processor, I can't help, I don't own one.

kiozen commented 2 years ago

That's fine. I will wait with officially releasing the binary until the weekend. Please test as much as possible.

BP22190 commented 2 years ago

Hello,

This binary version is not yet ready. After a few tries, some libraries cannot be loaded, because they depend on an absolute path of whoever compiled. For example :

dyld: Library not loaded: /Users/leon/Desktop/QMSDEVDIR/gdal-3.4.1/libgdal.dylib

I will try to recreate a binary myself with the updated manual. But there must be a problem in the building procedure to arrive at this error (in this example, the library is indeed in the application, but it is not it which is referenced).

Best regards,

kiozen commented 2 years ago

Is this an issue with the binaries I provided or an issue with you local build?

And if it is with the binaries can you please provide a bit more details? This is quite cryptic so far.

BP22190 commented 2 years ago

With the binaries you provided. Binaries from user "leon", it's not me.

BP22190 commented 2 years ago

Hello,

Some news:

Best regards,

BP22190 commented 2 years ago

Latest news:

I ended up succeeding in producing a binary .app file (it's a container under macOS). For this, it is necessary to review a certain number of points in the "howtobuildosx".

But... if I run this application, it crashes immediately! I just have a window that opens and it crashes afterwards. I tried with the -f option to see the log file, but there is no message that indicates the problem inside.

So I don't know what to do again...

Best regards,

BP22190 commented 2 years ago

@kiozen:

1) the binaries you provided to me seem to work fine, with the exception of gdal based tools for which the libraries are poorly linked.

2) on my side I did not succeed in rebuilding an executable version which does not crash: I would be very interested to know the exact procedure which allowed to create the binaries which you provided to me!

3) for my use, I changed strategy: rather than recompiling an executable, I patched those which you provided to me to correct the defective links. It works well that way.

I give you here : http://u.pc.cd/0WR7 the three archives equivalent to yours, but patched and functional. I added the necessary script to patch the binaries.

For my use, it's OK now (unless I find another bug later)... till next release... so I would have preferred to be able to recreate these binaries myself, so if you can find me the exact procedure... knowing that to do this, I use a clean macOS installation in a dedicated virtual machine. It seems to me from the binaries (and earlier ones for that matter) that the developers haven't been careful to remove the dependencies from their own machines.

Best regards,

kiozen commented 2 years ago

Just for the records: These binaries are not mine :wink: I just marshal between a person not able to communicate via that ticket and the ticket. :facepalm:

Anyways, I pushed new updates for the bundle scripts to

https://github.com/kiozen/qmapshack/tree/QMS-497

And here a re links to new binaries:

for 10.15 https://magentacloud.de/s/RrRx9iHzp4LzLK4

for 12.2: https://magentacloud.de/s/9Yn7BE6LZcCJs7x

Hope that makes sense. :sweat_smile:

Could you guys verify if these binaries work and if the build instructions and build scripts are now complete. I still hope we can get everyone into the position to compile MacOS bundles.

BP22190 commented 2 years ago

Just for the records: These binaries are not mine 😉 I just marshal between a person not able to communicate via that ticket and the ticket. 🤦

Yes, I understood !

Anyways, I pushed new updates for the bundle scripts to https://github.com/kiozen/qmapshack/tree/QMS-497

The "HowToBuildOSX.txt" don't works for me...

Is it possible to know the process of the "person not able to communicate via that ticket" ?

And here a re links to new binaries: for 10.15 https://magentacloud.de/s/RrRx9iHzp4LzLK4 for 12.2: https://magentacloud.de/s/9Yn7BE6LZcCJs7x

OK, no more linking problems with them.

Could you guys verify if these binaries work

Yes, they works for me.

and if the build instructions and build scripts are now complete.

No, don't works for me... (don't know if it's because of the instructions or scripts...).

Best regards,

kiozen commented 2 years ago

Is it possible to know the process of the "person not able to communicate via that ticket" ?

That's what is written down in the Howto and the scripts. But I would expect missing details, error and glitches. This is what usually happens if instructions and scripts are not cross tested by several persons.

sylanc3 commented 2 years ago

And here a re links to new binaries: for 10.15 https://magentacloud.de/s/RrRx9iHzp4LzLK4 for 12.2: https://magentacloud.de/s/9Yn7BE6LZcCJs7x

Links are dead, can anyone provide binaries? Thanks in advance!

d029940 commented 2 years ago

I tried to compile as proposed by https://github.com/kiozen/qmapshack/tree/QMS-497 I used Routino from MacPorts, since I could not build it as described ( I copied the files to the proposed structure) In the description adding qt5-qtwebengine was missing. So I took it from Macports. Since this was starting the qt5-qtwebengine compilation which takes hours and still is not completed, I took the qt5 libs from the official qt-installer. I was able to compile and build qmapshack , but on start, nothing happened.

I found that in QMapShack/Contents/Frameworks lot of frameworks were missing. I copied these frameworks from a QMapShack binary, and QMapShack worked. It seems to be that the missing frameworks are located in /opt/local/lib. Any idea? These are the missing frameworks: QtPdf.framework libIex-3_1.30.dylib libIlmThread-3_1.30.dylib libImath-3_1.29.dylib libOpenEXR-3_1.30.dylib libOpenEXRCore-3_1.30.dylib libOpenEXRUtil-3_1.30.dylib libaom.3.dylib libbrotlicommon.1.dylib libbrotlidec.1.dylib libbrotlienc.1.dylib libbz2.1.0.dylib libcrypto.3.dylib libcurl.4.dylib libdbus-1.3.dylib libde265.0.dylib libfreetype.6.dylib libgdal.dylib libgraphite2.3.dylib libharfbuzz.0.dylib libheif.1.dylib libidn2.0.dylib libjasper.4.dylib libjpeg.8.dylib libjpeg.9.dylib libjxl.0.6.dylib liblcms2.2.dylib liblz4.1.dylib liblzma.5.dylib libmng.2.dylib libmysqlclient.18.dylib libopenjp2.7.dylib libpcre2-8.0.dylib libpng16.16.dylib libpsl.5.dylib libquazip1-qt5.1.0.0.dylib libquazip1-qt5.1.3.0.dylib libroutino.dylib libsqlite3.0.dylib libssl.3.dylib libunistring.2.dylib libvmaf.1.dylib libwebp.7.dylib libwebpdemux.2.dylib libwebpmux.3.dylib libx265.199.dylib libxml2.2.dylib

kiozen commented 2 years ago

QMapShack has several dependencies like GDAL, PROJ, Quazip. These dependencies need to be installed in the system. I do not know the correct way for MacOS. On Linux they are installed in the usual library and data paths (e.g. /usr/lib) Especially GDAL and PROJ consist of many files. Dynamic libraries, executables and database files. They all have to be in the right locations to make GDAL and PROJ work as expected.

On Linux you can compile the packages and install them with make install. Or you simply install them via your package manager. On Windows all these files are stored within the QMapShack folder and the environment variable for GDAL and PROJ are set accordingly to make it work.

I can't tell how to do it the right way in MacOS. On Linux there is a commandline tool called ldd to check of an executable misses some dynamic libraries on startup. Maybe this tool is also available on MacOS. You can also try to start QMapShack form the console with -d as option to print some debug messages. Maybe that provides a hint about what is wrong.

d029940 commented 2 years ago

I am not sure if I truly found the source of the problem, but it turns out, that symlink files in the app-bundle were not copied with bundle-step. e.g. you will find libquazip1-qt5.1.3.0.dylib (or another version), but you will not find libquazip1-qt5.1.3.dylib and libquazip1-qt5.dylib, which are just links to libquazip1-qt5.1.3.0.dylib Solution: copy all lib files from the usr/lib folder created by PROJ, GDAL, QUAZIP in the Frameworks folder of the app bundle. Also for MacOS it is better to take ROUTINO from MacPorts.

I adapted the HowtoBuildOSX.txt file for review HowtoBuildOSX-d029940.txt

d029940 commented 2 years ago

I managed to build QMapShack for Mac M1 (ARM architecture).

Attached some helper files and the description HowtoBuildOSX-d029940 updated.txt QMS_build_files.zip

The files are stripped down to only build a Mac version of QMapShack.

In addition a link to the app built for Mac M1 QMapShack 1.16.1 for Mac M1

sylanc3 commented 2 years ago

Hello @d029940 , thanks for providing the binary. Unfortunately it isn't working on my M1 machine.

Thanks again.

sylanc3 commented 2 years ago

After installing https://libgeos.org by using Homebrew (brew install geos) it works! Thanks a lot!

d029940 commented 2 years ago

Thank you for your feedback. I will copy libgoes into the bundle. I think geos is a dependency of gdal

kiozen commented 2 years ago

Hey that's great news. If you think everything is fine it would be nice to get all your changes via PR into the code base.

If you want I can publish the resulting binary bundle in the release section of the project.

d029940 commented 2 years ago

Hello, the binary is already updated [QMapShack 1.16.1 for Mac M1](https://drive.google.com/drive/folders/1Tm6X8zBvyaNROMzxnub2YEXZzPjxKLEl?usp=sharing)

Hope that all dependencies are now included.

For PR, this will take some time, because I have to learn about PRs first. (I'm not a GitHub-guy yet) Mainly, there are some helper files and the description on how to build QMapShack. This interferes a bit with your approach: Your are using Macports, myself I am using Homebrew. The two approaches are valid, so I look at my approach as an alternative and not a replacement.

kiozen commented 2 years ago

Either way is fine for me. But it should be documented. I experienced on your own how frustrating missing or incorrect documentation is. And it would be nice to have the process automated by shell scripts. By that it's easier to pass binary creation to someone else.

About committing code there is an article in the Wiki: https://github.com/Maproom/qmapshack/wiki/DeveloperCommitCode

And of course I can help if there are any questions.

kiozen commented 2 years ago

I have uploaded the binary to the release area

d029940 commented 2 years ago

Please upload the binary again. I found a (severe) error in the bundle files in QMapShack/MacOSX. Refer to uploaded patches. Without applying these patches and taking GDAL from Homwbrew / MacPorts bundling will include all files of the bin directory of those package manager. The least you will increase the size of the app bundle. Moreover, I think this is a security issue. qmapshack-bundle-patch.zip

I also created a script where I can build QMapShack for the arm64 and x86_64 architecture with almost no manual intervention.

I try to look at the https://github.com/Maproom/qmapshack/wiki/DeveloperCommitCode over the weekend, in order to provide proper code commitments.

kiozen commented 2 years ago

can you please apply everything, test it and provide a new zip.

d029940 commented 2 years ago

Yes, but over the weekend.

Manfred Kern EMail: @.***

On 27. Jul 2022, at 07:30, Kiozen @.***> wrote:

 can you please apply everything, test it and provide a new zip.

— Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you were mentioned.