search

mapnik / node-mapnik

Bindings to mapnik for node.js
http://mapnik.org/documentation/node-mapnik
BSD 3-Clause "New" or "Revised" License
533 stars 165 forks source link

Enable all platforms again #976

Open mathisloge opened 3 years ago

mathisloge commented 3 years ago

So this PR adds a way to build for all platforms that are supported by mapnik (including windows) Currently the whole dependency setup is done via vcpkg. I recommend this for windows, because, as #848 stated, it would take a lot of effort to pull in all the dependencies with a script. On linux and osx the system libraries should be used. This splitting is generally easy. On windows the submodule would be cloned or, as it is also possibile, added via FetchContent. Otherwise mapnik would be found via pkg-config or cmakes find_package.

However since this will use cmake as a build system i had to remove node-gyp. I think it will need discussion if you want to pursude this path. This will use cmake-js as the build system and node-pre-gyp for packaging and publishing.

Depends on:

if someone wants to try this PR, the file build\vcpkg_installed\<your_triplet>\share\mapnik\MapnikTargets.cmake needs a patch at line 57 with [...]vcpkg_installed/include to [...]vcpkg_installed/<your_triplet>/include. You will need to build manually with cmake-js build--CDCMAKE_TOOLCHAIN_FILE="./vcpkg/scripts/buildsystems/vcpkg.cmake" --target node-mapnik and after the first build file, apply the described workaround. RESOLVED

Todo:

prebuild binaries

For development and testing purposes, there is a devbuild npm package: https://www.npmjs.com/package/@mathisloge/mapnik

Currently there should be published packages for windows and linux.

integration tests

Node.js CI Simple integration test with windows and linux each with node versions 12, 14 and 16 https://github.com/mathisloge/node-mapnik-test

mathisloge commented 3 years ago

I will wait for some feedback before putting more effort into it.

mathisloge commented 3 years ago

mainly seeking some feedback from you @artemp :D

GimpMaster commented 3 years ago

Just curious if this might be merged anytime in the near future?

mathisloge commented 3 years ago

@GimpMaster I'm first need to merge https://github.com/mapnik/mapnik/pull/4252 as that resolves some issues. Currently ogr and gdal plugins can't be loaded. I don't know exactly what the reason is, because all dependencies are resolved correctly and a normal mapnik application can also load those plugins. So I need to investigate that further.

GimpMaster commented 3 years ago

thank you @mathisloge - Could I just npm add your git repo and try on windows? or is there more to it than that?

mathisloge commented 3 years ago

@GimpMaster I know that @artemp haven't pushed proj6 to master yet. So expect that a lot of tests would fail. Also i haven't test all things thoroughly, since it is very much a WIP.

However, if you don't need gdal or ogr, you could clone the repo (and submodules!) and perform the needed install steps. It should be as little as:

You need a compiler for c++, preferably msvc 2019 and cmake >= 3.21, 3.20 might also be ok. Havent tested the lower versions yet.

I would appreciate any feedback if you build and use it :)

mathisloge commented 3 years ago

@GimpMaster just tested it to install just with npm install https://github.com/mathisloge/node-mapnik this does not download the submodules. However you could also clone https://github.com/mathisloge/node-mapnik and run npm install and npm run build. After that you should be able to add it to your project with npm install <path_to_node-mapnik>. (At least that worked for me)

GimpMaster commented 3 years ago

Great,

I'll give it a try tonight.

I'll give any feedback I find.

Thanks!

On Mon, Oct 11, 2021 at 2:50 PM Mathis @.***> wrote:

@GimpMaster https://github.com/GimpMaster just tested it to install just with npm install https://github.com/mathisloge/node-mapnik this does not download the submodules. However you could also clone https://github.com/mathisloge/node-mapnik and run npm install and npm run build. After that you should be able to add it to your project with npm install .

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/mapnik/node-mapnik/pull/976#issuecomment-940431225, or unsubscribe https://github.com/notifications/unsubscribe-auth/AB3UXWEZ56E5TAWGYKISV5LUGNE2HANCNFSM47W574WA . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

GimpMaster commented 3 years ago

@mathisloge - Well got closer. Installed your repo. initialized git sub modules. Updated CMake to latest version. Had to also install Windows SDK 8.1. Then finally onto vcpkg.

vcpkg installed 136 packages, compiled for 1.5 hours! and on the last one mapnik:x64-windows it failed

Error: Building package mapnik:x64-windows failed with: BUILD_FAILED Please ensure you're using the latest portfiles with .\vcpkg update, then submit an issue at https://github.com/Microsoft/vcpkg/issues including: Package: mapnik:x64-windows Vcpkg version: 2021-09-10-2059ef11aa6067e6f59b0d939c5d17e3c5c47d3e

Still trying to look through logs why that failed.

GimpMaster commented 3 years ago

looking at the log this is the error I see error C2039: 'min': is not a member of 'std'

Trying to compile with VS2017

mathisloge commented 3 years ago

Ah okay. Yeah I think usera had reported issues with <vs2019 previously. Since the windows headers are defining min and max as macros and collide with anything that named min or max.

GimpMaster commented 3 years ago

Are you using VS2019?

I thought node installs typically install VS2017 for node-gyp

On Mon, Oct 11, 2021 at 23:55 Mathis @.***> wrote:

Ah okay. Yeah I think usera had reported issues with <vs2019 previously. Since the windows headers are defining min and max as macros and collide with anything that named min or max.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/mapnik/node-mapnik/pull/976#issuecomment-940687868, or unsubscribe https://github.com/notifications/unsubscribe-auth/AB3UXWE2DFNW3TJTAVKRC7DUGPEW5ANCNFSM47W574WA . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

mathisloge commented 3 years ago

If you have a chance to update to vs2019 that would be the fastest way.

I could add a compile define NOMINMAX to mapnik for <vs2019. But that would need some testing from my side, as I would need to test with vs2017 etc. if there aren't any dependencies that depend on the macro.

mathisloge commented 3 years ago

I'm using vs2019. This PR uses node-pre-gyp just for the packaging. The build will be invoked through cmake-js

GimpMaster commented 3 years ago

I think it would be good to get it working on vs2017. I think when you default install NodeJS LTS on windows through the MSI installer it installs vs2017.

On Tue, Oct 12, 2021 at 00:07 Mathis @.***> wrote:

I'm using vs2019. This PR uses node-pre-gyp just for the packaging. The will be invoked through cmake-js

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/mapnik/node-mapnik/pull/976#issuecomment-940693722, or unsubscribe https://github.com/notifications/unsubscribe-auth/AB3UXWBPCVX2SYWKGWX4XLDUGPGCJANCNFSM47W574WA . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

mathisloge commented 3 years ago

I will look into it. But it will be at a later time as I first need to get all other things done. So currently only vs2019 and vs2022. But vs2017 is a valid target and needs to be fixed

GimpMaster commented 3 years ago

Anything I can do to help? I’m not familiar with node-gyp but I was a c++ developer for 10 years however the last six has been NodeJS and react.

On Tue, Oct 12, 2021 at 09:24 Mathis @.***> wrote:

I will look into it. But it will be at a later time as I first need to get all other things done. So currently only vs2019 and vs2022. But vs2017 is a valid target and needs to be fixed

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/mapnik/node-mapnik/pull/976#issuecomment-941117829, or unsubscribe https://github.com/notifications/unsubscribe-auth/AB3UXWD57WIU2S6L3BORFJTUGRHMTANCNFSM47W574WA . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

mathisloge commented 3 years ago

Just to note: The build isn't invoked with node-gyp. Instead it uses cmake. Any limitations related with node-gyp don't apply here.

So the mapnik build error is related to mapnik itself. Someone had already begin with vs2017. https://github.com/mapnik/mapnik/pull/4184 But I don't know if those are really the only two occurances where the macros would collide with the min/max usage.

I'm working currently on https://github.com/mapnik/mapnik/pull/4252 which fixes some problems i have in this PR.

And then the outstanding leftover issues with this are the plugins ogr and gdal which can't be loaded.

GimpMaster commented 3 years ago

@mathisloge - I played around a little bit more with this tonight trying to get 2017 working. As far as I can tell right now the only two std::min / std::max errors are in

include/mapnik/util/char_array_buffer.hpp

line 74, 81, 82, 83

current_ = std::min(begin_ + off, end_)

I tried manually modifying this file but every time I do an npm run build vcpkg gets a clean copy and overwrites my local changes.

Any thoughts on how I can change vcpkg to pull from a forked repo for mapnik?

Thanks!

mathisloge commented 3 years ago

@GimpMaster the quickest way would be to modify vcpkg/ports/mapnik/portfile.cmake. Just replace 

vcpkg_from_github(
    OUT_SOURCE_PATH SOURCE_PATH
    REPO mapnik/mapnik
    REF 0edb018465790cd156d6849557fa7fd568755ebb
    SHA512 cd6180b96dbfbc1bafbc8f2a4ff2091a0e8c0c42b0569ef83640ad1f5766ff74ea95f8c6cef53f879396cbceebc6396c1a977676eca67c401b8e145a9ceae7e2
    HEAD_REF master
)

with set(SOURCE_PATH <absolute path to your local fork>)

or if you want to use your github repo, replace REPO mapnik/mapnik with REPO <your_name>/mapnik But you would need to update the REF to the latest commit and SHA512. The expected SHA512 will be printed out when building. So you could first change REPO and REF, build, copy expected SHA512 and replace it and build again. (The first build will fail after the download, so no long wait times :) )

GimpMaster commented 3 years ago

So I went with your suggestion of set(SOURCE_PATH )

I know it is correctly pointing to the path because the first time I had the path wrong and it failed immediately. I then fixed the absolute path and it takes a lot longer but ultimately fails. It looks like it doesn't even try to compile. It runs all the cmake checks to make sure it has all the libraries but then it ultimately ends saying

Build files have been written to ....

This is where I get to when running npm install.

See attached config-x64-windows-out.log file

install-x64-windows-dbg-out.log

Detecting compiler hash for triplet x64-windows...
The following packages will be built and installed:
    mapnik[cairo,core,grid-renderer,input-csv,input-gdal,input-geobuf,input-geojson,input-ogr,input-pgraster,input-postgis,input-raster,input-shape,input-sqlite,input-topojson,jpeg,png,proj,svg-renderer,tiff,utility-mapnik-index,utility-shapeindex,webp]:x64-windows -> 2021-10-06
Restored 0 packages from C:\Users\Josh\AppData\Local\vcpkg\archives in 1.209 ms. Use --debug to see more details.
Starting package 1/1: mapnik:x64-windows
Building package mapnik[cairo,core,grid-renderer,input-csv,input-gdal,input-geobuf,input-geojson,input-ogr,input-pgraster,input-postgis,input-raster,input-shape,input-sqlite,input-topojson,jpeg,png,proj,svg-renderer,tiff,utility-mapnik-index,utility-shapeindex,webp]:x64-windows...
-- Configuring x64-windows
-- Building x64-windows-dbg
CMake Error at scripts/cmake/vcpkg_execute_build_process.cmake:155 (message):
    Command failed: "C:/Program Files/CMake/bin/cmake.exe" --build . --config Debug --target install -- -v -j9
    Working Directory: H:/Fast_Dev/imsar/node-mapnik/vcpkg/buildtrees/mapnik/x64-windows-dbg
    See logs for more information:
      H:\Fast_Dev\imsar\node-mapnik\vcpkg\buildtrees\mapnik\install-x64-windows-dbg-out.log

Call Stack (most recent call first):
  H:/Fast_Dev/imsar/node-mapnik/build/vcpkg_installed/x64-windows/share/vcpkg-cmake/vcpkg_cmake_build.cmake:102 (vcpkg_execute_build_process)
  H:/Fast_Dev/imsar/node-mapnik/build/vcpkg_installed/x64-windows/share/vcpkg-cmake/vcpkg_cmake_install.cmake:41 (vcpkg_cmake_build)     
  ports/mapnik/portfile.cmake:65 (vcpkg_cmake_install)
  scripts/ports.cmake:141 (include)

Error: Building package mapnik:x64-windows failed with: BUILD_FAILED
Please ensure you're using the latest portfiles with `.\vcpkg update`, then
submit an issue at https://github.com/Microsoft/vcpkg/issues including:
  Package: mapnik:x64-windows
  Vcpkg version: 2021-09-10-2059ef11aa6067e6f59b0d939c5d17e3c5c47d3e

Additionally, attach any relevant sections from the log files above.

I'm not sure how to get it passed this step and to actually try compiling my local fork. Appreciate any next steps to try.

mathisloge commented 3 years ago

It looks like it is still not correctly wrapped. Have you wrapped them like this: (std::min)(x, y); ? so that the function-like macro won't be applied? 

H:\Fast_Dev\imsar\mapnik\include\mapnik/util/char_array_buffer.hpp(74): error C2039: 'min': is not a member of 'std'
C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Tools\MSVC\14.16.27023\include\iterator(16): note: see declaration of 'std'
H:\Fast_Dev\imsar\mapnik\include\mapnik/util/char_array_buffer.hpp(74): error C3861: 'min': identifier not found
H:\Fast_Dev\imsar\mapnik\include\mapnik/util/char_array_buffer.hpp(81): error C2039: 'min': is not a member of 'std'
C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Tools\MSVC\14.16.27023\include\iterator(16): note: see declaration of 'std'
H:\Fast_Dev\imsar\mapnik\include\mapnik/util/char_array_buffer.hpp(81): error C3861: 'min': identifier not found
H:\Fast_Dev\imsar\mapnik\include\mapnik/util/char_array_buffer.hpp(82): error C2039: 'min': is not a member of 'std'
C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Tools\MSVC\14.16.27023\include\iterator(16): note: see declaration of 'std'
H:\Fast_Dev\imsar\mapnik\include\mapnik/util/char_array_buffer.hpp(82): error C3861: 'min': identifier not found
H:\Fast_Dev\imsar\mapnik\include\mapnik/util/char_array_buffer.hpp(83): error C2039: 'max': is not a member of 'std'
C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Tools\MSVC\14.16.27023\include\iterator(16): note: see declaration of 'std'
H:\Fast_Dev\imsar\mapnik\include\mapnik/util/char_array_buffer.hpp(83): error C3861: 'max': identifier not found

Is this H:\Fast_Dev\imsar the right repo?

GimpMaster commented 3 years ago

@mathisloge - I was able to get mapnik to build with VS2017. The issue wasn't the macro the issue was that std::min std::max is defined in

#include <algorithm>

I just needed to add the #include into two places.

src/webp_reader.cpp include/mapnik/util/char_array_buffer.hpp

After that it built successfully.

Currently trying to yarn add your node-mapnik to my project. Feels like its rebuilding it again now instead of using the binaries, but my CPU is humming so it must be building again.

GimpMaster commented 3 years ago

@mathisloge - So that all worked! I was able to install and use mapnik on windows with VS2017 compiling.

However I didn't realize that you had already mentioned this but I DO need gdal to work. I'm rastering geotiffs which uses the gdal library for rendering. I'm sure you have already seen this error

Mapnik LOG> 2021-10-15 13:58:43: Problem loading plugin library: H:\Fast_Dev\imsar\mapnik-test\node_modules\mapnik\lib\binding\napi-v3\mapnik\input\gdal.input (dlopen failed - plugin likely has an unsatisfied dependency or incompatible ABI)

Let me know if I can help on any of that too.

mathisloge commented 3 years ago

Yeah. I'm working on it. I have narrowed it down to libpq. Need to attach windbg to it. To get more infos, why it fails to load and which symbols are missing.

Sorry for the inconvenience!

GimpMaster commented 3 years ago

@mathisloge - No inconvenience at all. You have been very helpful!

Two quick questions: When this is all done will there be a binary version that we can distribute so others don't need to go through all the build steps?

Do you want to submit the MR for mapnik for VS2017 compiling or do you want me to? Seems like you have some prior experience with them doing so, but I would be glad to if you don't.

mathisloge commented 3 years ago

I try to get it ready as fast as possible. But I don't know how long I need to debug. The pr is ready to distribute binaries (npm run publish).

You can open the PR if you want :)

mathisloge commented 3 years ago

@GimpMaster https://github.com/microsoft/vcpkg/pull/20824 vcpkg will soon be updated and has your fix.

GimpMaster commented 3 years ago

Awesome. GDAL is still not working correct? @mathisloge

mathisloge commented 3 years ago

Yeah. I guess it is something with libpq. I haven't had so much time this weekend. hopefully I get some results in this week.

mathisloge commented 3 years ago

@GimpMaster Okay nvm all my comments regarding libpq. After trying to debug node itself which was an awful experience to say. I've just written a small test program to load different dlls and the plugins. So there isn't a real dependency issue since all plugins etc. can actually be loaded. I've noticed, that if you run a script with node inside lib/binding/napi-v3 it actually loads gdal. so the dlls just wasn't in the search path. Added in https://github.com/mapnik/node-mapnik/pull/976/commits/dcc0517616a4d8f4cf1518b97c6ef7a612e276b2

Arghhh... sometimes it's really just the obvious that I'm not paying attention to.

mathisloge commented 3 years ago

Just to note: You also might need to add the proj6 branch from https://github.com/mapbox/mapnik-vector-tile/tree/proj6 for mapnik-vector-tile.

mathisloge commented 3 years ago

So you should be able to render raster tiles now 👍 map

GimpMaster commented 3 years ago

Your right it does work. I'm not sure why I was seeing the problem before. All I did was a yarn link in the local built version and then linked in in my test project and it used gdal properly.

If I wanted to distribute this to my co-workers without them having to eat up 30+ Gigs of disk space and hours of compiling would I just give them all the files/folders under the lib directory? Or what would be the easiest way to do that? Assuming they are on windows using same node version.

mathisloge commented 3 years ago

Yes, dcc0517 is adding the search path by default. (Sorry. In my comment above I didn't mean you but me :D )

I think it should be enough to share the lib folder with your colleagues.

mathisloge commented 3 years ago

So now there is the support to build the cmake project with node-pre-gyp. I've replaced the whole binding.gyp and created two custom targets instead. This also enables the node-pre-gyp feature of specifying multiple n-api versions and let me drop cmake-js.

Works with windows and ubuntu. I've not tested this on mac yet

mathisloge commented 3 years ago

I've now ported mapnik-vector-tile to cmake. So this dependency can be removed from the package.json and consumed directly via cmake if a build is required.

https://github.com/mapbox/mapnik-vector-tile/compare/master...mathisloge:cmake

I will open a PR if the PROJ6 changes are merged

GimpMaster commented 3 years ago

@mathisloge - I noticed some more updates on this. Just curious how close you are to complete and pushing binaries to S3 for windows?

mathisloge commented 3 years ago

@GimpMaster It depends when I'm getting to solve the issues with UTF-8 file naming.

However, it is mostly ready, but I don't plan on publishing any packages via s3 since I don't have any instance of it and don't plan to get one. But I could look into publishing binaries to github.

mathisloge commented 2 years ago

@GimpMaster So I've added github publishing for the build process. There should be windows and linux prebuild packages for https://www.npmjs.com/package/@mathisloge/mapnik

3nprob commented 2 years ago

This image does not build for me (linux, aarch64, built base image from https://github.com/mapnik/mapnik/pull/4267/commits/7631f93f5449aaf0aa041b44f69f2cae66aa117e):

Step 6/7 : RUN npm install && rm -rf build*
 ---> Running in 700a0d4f20f4
npm WARN deprecated querystring@0.2.0: The querystring API is considered Legacy. new code should use the URLSearchParams API instead.
npm WARN deprecated uuid@3.4.0: Please upgrade  to version 7 or higher.  Older versions may use Math.random() in certain circumstances, which is known to be problematic.  See https://v8.dev/blog/math-random for details.

> @mathisloge/mapnik@5.0.0-devbuild10 install
> node-pre-gyp install --fallback-to-build

node-pre-gyp info it worked if it ends with ok
node-pre-gyp info using node-pre-gyp@1.0.9
node-pre-gyp info using node@12.21.0 | linux | arm64
node-pre-gyp info check checked for "/nodemapnik/lib/binding/napi-v3/mapnik.node" (not found)
node-pre-gyp http GET https://github.com/mathisloge/node-mapnik/releases/download/v5.0.0-devbuild10/linux-arm64-napi-v3.tar.gz
node-pre-gyp ERR! install response status 404 Not Found on https://github.com/mathisloge/node-mapnik/releases/download/v5.0.0-devbuild10/linux-arm64-napi-v3.tar.gz
node-pre-gyp WARN Pre-built binaries not installable for @mathisloge/mapnik@5.0.0-devbuild10 and node@12.21.0 (node-v72 ABI, glibc) (falling back to source compile with node-gyp)
node-pre-gyp WARN Hit error response status 404 Not Found on https://github.com/mathisloge/node-mapnik/releases/download/v5.0.0-devbuild10/linux-arm64-napi-v3.tar.gz
gyp info it worked if it ends with ok
gyp info using node-gyp@7.1.2
gyp info using node@12.21.0 | linux | arm64
gyp info ok
gyp info it worked if it ends with ok
gyp info using node-gyp@7.1.2
gyp info using node@12.21.0 | linux | arm64
gyp info find Python using Python version 3.9.5 found at "/usr/bin/python3"
gyp info spawn /usr/bin/python3
gyp info spawn args [
gyp info spawn args   '/usr/share/nodejs/node-gyp/gyp/gyp_main.py',
gyp info spawn args   'binding.gyp',
gyp info spawn args   '-f',
gyp info spawn args   'make',
gyp info spawn args   '-I',
gyp info spawn args   '/nodemapnik/build/config.gypi',
gyp info spawn args   '-I',
gyp info spawn args   '/usr/share/nodejs/node-gyp/addon.gypi',
gyp info spawn args   '-I',
gyp info spawn args   '/usr/include/nodejs/common.gypi',
gyp info spawn args   '-Dlibrary=shared_library',
gyp info spawn args   '-Dvisibility=default',
gyp info spawn args   '-Dnode_root_dir=/usr/include/nodejs',
gyp info spawn args   '-Dnode_gyp_dir=/usr/share/nodejs/node-gyp',
gyp info spawn args   '-Dnode_lib_file=/usr/include/nodejs/<(target_arch)/node.lib',
gyp info spawn args   '-Dmodule_root_dir=/nodemapnik',
gyp info spawn args   '-Dnode_engine=v8',
gyp info spawn args   '--depth=.',
gyp info spawn args   '--no-parallel',
gyp info spawn args   '--generator-output',
gyp info spawn args   'build',
gyp info spawn args   '-Goutput_dir=.'
gyp info spawn args ]
gyp info ok
gyp info it worked if it ends with ok
gyp info using node-gyp@7.1.2
gyp info using node@12.21.0 | linux | arm64
gyp info spawn make
gyp info spawn args [ 'BUILDTYPE=Release', '-C', 'build' ]
make: Entering directory '/nodemapnik/build'
  ACTION configuring node-mapnik... build/CMakeCache.txt
CMake Error at /usr/share/cmake-3.23/Modules/CMakeDetermineSystem.cmake:130 (message):
  Could not find toolchain file:
  /nodemapnik/vcpkg/scripts/buildsystems/vcpkg.cmake
Call Stack (most recent call first):
  CMakeLists.txt:3 (project)

CMake Error: CMake was unable to find a build program corresponding to "Unix Makefiles".  CMAKE_MAKE_PROGRAM is not set.  You probably need to select a different build tool.
CMake Error: CMAKE_C_COMPILER not set, after EnableLanguage
CMake Error: CMAKE_CXX_COMPILER not set, after EnableLanguage
-- Configuring incomplete, errors occurred!
make: *** [build-node-mapnik.target.mk:13: build/CMakeCache.txt] Error 1
make: Leaving directory '/nodemapnik/build'
gyp ERR! build error
gyp ERR! stack Error: `make` failed with exit code: 2
gyp ERR! stack     at ChildProcess.onExit (/usr/share/nodejs/node-gyp/lib/build.js:194:23)
gyp ERR! stack     at ChildProcess.emit (events.js:314:20)
gyp ERR! stack     at Process.ChildProcess._handle.onexit (internal/child_process.js:276:12)
gyp ERR! System Linux 5.18.0-4-MANJARO-ARM
gyp ERR! command "/usr/bin/node" "/usr/share/nodejs/node-gyp/bin/node-gyp.js" "build" "--fallback-to-build" "--module=/nodemapnik/lib/binding/napi-v3/mapnik.node" "--module_name=mapnik" "--module_path=/nodemapnik/lib/binding/napi-v3" "--napi_version=7" "--
node_abi_napi=napi" "--napi_build_version=3" "--node_napi_label=napi-v3"
gyp ERR! cwd /nodemapnik
gyp ERR! node -v v12.21.0
gyp ERR! node-gyp -v v7.1.2
gyp ERR! not ok
node-pre-gyp ERR! build error
node-pre-gyp ERR! stack Error: Failed to execute '/usr/bin/node /usr/share/nodejs/node-gyp/bin/node-gyp.js build --fallback-to-build --module=/nodemapnik/lib/binding/napi-v3/mapnik.node --module_name=mapnik --module_path=/nodemapnik/lib/binding/napi-v3 --n
api_version=7 --node_abi_napi=napi --napi_build_version=3 --node_napi_label=napi-v3' (1)
node-pre-gyp ERR! stack     at ChildProcess.<anonymous> (/nodemapnik/node_modules/@mapbox/node-pre-gyp/lib/util/compile.js:89:23)
node-pre-gyp ERR! stack     at ChildProcess.emit (events.js:314:20)
node-pre-gyp ERR! stack     at maybeClose (internal/child_process.js:1022:16)
node-pre-gyp ERR! stack     at Process.ChildProcess._handle.onexit (internal/child_process.js:287:5)
node-pre-gyp ERR! System Linux 5.18.0-4-MANJARO-ARM
node-pre-gyp ERR! command "/usr/bin/node" "/nodemapnik/node_modules/.bin/node-pre-gyp" "install" "--fallback-to-build"
node-pre-gyp ERR! cwd /nodemapnik
node-pre-gyp ERR! node -v v12.21.0
node-pre-gyp ERR! node-pre-gyp -v v1.0.9
node-pre-gyp ERR! not ok
Failed to execute '/usr/bin/node /usr/share/nodejs/node-gyp/bin/node-gyp.js build --fallback-to-build --module=/nodemapnik/lib/binding/napi-v3/mapnik.node --module_name=mapnik --module_path=/nodemapnik/lib/binding/napi-v3 --napi_version=7 --node_abi_napi=n
api --napi_build_version=3 --node_napi_label=napi-v3' (1)
npm notice
npm notice New major version of npm available! 7.5.2 -> 8.12.1
npm notice Changelog: <https://github.com/npm/cli/releases/tag/v8.12.1>
npm notice Run `npm install -g npm@8.12.1` to update!
npm notice
npm ERR! code 1
npm ERR! path /nodemapnik
npm ERR! command failed
npm ERR! command sh -c node-pre-gyp install --fallback-to-build

npm ERR! A complete log of this run can be found in:
npm ERR!     /root/.npm/_logs/2022-06-03T02_14_10_285Z-debug.log

Pre-built binaries should not be required when building from source.

3nprob commented 2 years ago

Trying to manually patch it up by adding '-DCMAKE_C_COMPILER=gcc', '-DCMAKE_CXX_COMPILER=g++' to the actions in binding.gyp and removing the CMAKE_TOOLCHAIN_FILE declaration, we get some further progress but it stops soon after at:

-- Performing Test CMAKE_HAVE_LIBC_PTHREAD - Failed                                                                                                                                                                                                   [109/1905]
-- Looking for pthread_create in pthreads
-- Looking for pthread_create in pthreads - not found
-- Looking for pthread_create in pthread
-- Looking for pthread_create in pthread - found
-- Found Threads: TRUE
-- Found PkgConfig: /usr/bin/pkg-config (found version "0.29.2")
-- Checking for module 'cairo'
--   Found cairo, version 1.16.0
-- Checking for module 'proj'
--   Found proj, version 7.2.1
-- Found Protobuf: /usr/lib/aarch64-linux-gnu/libprotobuf.so;-lpthread (found version "3.12.4")
-- Found the following ICU libraries:
--   uc (required)
--   i18n (required)
--   data (required)
-- Found ICU: /usr/include (found version "67.1")
-- Found Boost: /usr/lib/aarch64-linux-gnu/cmake/Boost-1.74.0/BoostConfig.cmake (found suitable version "1.74.0", minimum required is "1.73") found components: filesystem system regex
-- Found Freetype: /usr/lib/aarch64-linux-gnu/libfreetype.so (found version "2.10.4")
-- Found LibXml2: /usr/lib/aarch64-linux-gnu/libxml2.so (found version "2.9.10")
-- Found ZLIB: /usr/lib/aarch64-linux-gnu/libz.so (found version "1.2.11")
-- Found PNG: /usr/lib/aarch64-linux-gnu/libpng.so (found version "1.6.37")
-- Found JPEG: /usr/lib/aarch64-linux-gnu/libjpeg.so (found version "80")
-- Found TIFF: /usr/lib/aarch64-linux-gnu/libtiff.so (found version "4.2.0")
-- Found WebP: /usr/lib/aarch64-linux-gnu/libwebp.so (found version "0.6.1")
-- Found the following ICU libraries:
--   uc (required)
--   i18n (required)
--   data (required)
-- Found ICU: /usr/include (found version "67.1")
CMake Error at /usr/lib/aarch64-linux-gnu/cmake/harfbuzz/harfbuzz-config.cmake:49 (add_library):
  add_library cannot create imported target "harfbuzz::harfbuzz" because
  another target with the same name already exists.
Call Stack (most recent call first):
  /usr/share/cmake-3.23/Modules/CMakeFindDependencyMacro.cmake:47 (find_package)
  /usr/local/lib/cmake/mapnik/mapnikConfig.cmake:28 (find_dependency)
  build/_deps/mapnik_vector_tile-src/CMakeLists.txt:24 (find_package)

CMake Error at /usr/lib/aarch64-linux-gnu/cmake/harfbuzz/harfbuzz-config.cmake:54 (add_library):
  add_library cannot create imported target "harfbuzz::icu" because another
  target with the same name already exists.
Call Stack (most recent call first):
  /usr/share/cmake-3.23/Modules/CMakeFindDependencyMacro.cmake:47 (find_package)
  /usr/local/lib/cmake/mapnik/mapnikConfig.cmake:28 (find_dependency)
  build/_deps/mapnik_vector_tile-src/CMakeLists.txt:24 (find_package)

CMake Error at /usr/lib/aarch64-linux-gnu/cmake/harfbuzz/harfbuzz-config.cmake:60 (add_library):
  add_library cannot create imported target "harfbuzz::subset" because
  another target with the same name already exists.
Call Stack (most recent call first):
  /usr/share/cmake-3.23/Modules/CMakeFindDependencyMacro.cmake:47 (find_package)
  /usr/local/lib/cmake/mapnik/mapnikConfig.cmake:28 (find_dependency)
  build/_deps/mapnik_vector_tile-src/CMakeLists.txt:24 (find_package)

CMake Error at /usr/lib/aarch64-linux-gnu/cmake/harfbuzz/harfbuzz-config.cmake:69 (add_library):
  add_library cannot create imported target "harfbuzz::gobject" because
  another target with the same name already exists.
Call Stack (most recent call first):
  /usr/share/cmake-3.23/Modules/CMakeFindDependencyMacro.cmake:47 (find_package)
  /usr/local/lib/cmake/mapnik/mapnikConfig.cmake:28 (find_dependency)
  build/_deps/mapnik_vector_tile-src/CMakeLists.txt:24 (find_package)

Same result even if I remove the one mention of harfbuzz in this branch in scripts/install_deps.sh

I'm not a cmake pro and not sure at all how to continue from here. Currently attempting from https://github.com/3nprob/node-mapnik/commit/6ba144f8de64b3e3dddf7049f179e4176f757e34 (https://github.com/mathisloge/node-mapnik/compare/master...3nprob:linux-docker-build?expand=1) and I can't see where harfbuzz would enter in the first,let alone second, place.

mathisloge commented 2 years ago

Hey, I'm currently not on my PC so I can't help you that much.

You can try to set -DCMAKE_DISABLE_FIND_PACKAGE_harfbuzz=OFF to force pkgconf to find harfbuzz when configuring mapnik. Unfortunately harfbuzz's own provided cmake config files are seriously broken on some distributions.

3nprob commented 2 years ago

You can try to set -DCMAKE_DISABLE_FIND_PACKAGE_harfbuzz=OFF to force pkgconf to find harfbuzz when configuring mapnik. Unfortunately harfbuzz's own provided cmake config files are seriously broken on some distributions.

Hmm, I tried setting that both here and here with same outcome. Would it go somewhere else?

wxitguy commented 2 years ago

Hello,

I was able to successfully build the latest devbuild10 Mapnik on my Windows 10 system.

My issue is that I am trying to build Mapnik to use in an Electron app for Windows. When I get to the point that I can successfully build Mapnik from source, I then run it through electron-rebuild it does appear to rebuild successfully but when starting my program I get "a dynamic link library (DLL) initialization routine failed" errors every time.

My electron version is 14.2.9 but I've tried several different versions of electron up to 18 with the same result.

I'm not sure what is missing or needs to be done at this point, any thoughts/ideas?

mathisloge commented 2 years ago

@3nprob oh sorry. You actually need to set DCMAKE_DISABLE_FIND_PACKAGE_harfbuzz to ON my fault.

@wxitguy glad to hear that you were successful compiling from source. IIRC you need to compile a different napi lib for electron. I'm currently using https://github.com/mathisloge/cmake-napi but it handles only the real node api bindings and not the one for electron. As I'm currently fairly busy with other things in my live, I won't be able to look into it before august. But I would be really happy, if you could report any progress on your site.