Closed sjwild closed 2 years ago
Odd. Is this with the latest released GDAL.jl versions on both 1.6 and 1.7?
I am using GDAL 1.2.6
(@v1.7) pkg> status GDAL
Status `~/.julia/environments/v1.7/Project.toml`
[add2ef01] GDAL v1.2.6
Ok and can you try it in a clean environment with only GDAL? And what is the output of versioninfo()
(are you on a M1 Mac?) Our tests currently pass on 1.3, 1.7 and nightly: https://github.com/JuliaGeo/GDAL.jl/actions/runs/1645923012.
Still get the same message. I am on an M1 mac.
julia> versioninfo()
Julia Version 1.7.0
Commit 3bf9d17731 (2021-11-30 12:12 UTC)
Platform Info:
OS: macOS (arm64-apple-darwin21.1.0)
CPU: Apple M1
WORD_SIZE: 64
LIBM: libopenlibm
LLVM: libLLVM-12.0.1 (ORCJIT, cyclone)
julia> using GDAL
ERROR: InitError: UndefVarError: libgdal not defined
Stacktrace:
[1] cplseterrorhandler
@ ~/.julia/packages/GDAL/LljeA/src/GDAL.jl:1949 [inlined]
[2] __init__()
@ GDAL ~/.julia/packages/GDAL/LljeA/src/GDAL.jl:33004
[3] _include_from_serialized(path::String, depmods::Vector{Any})
@ Base ./loading.jl:768
[4] _require_search_from_serialized(pkg::Base.PkgId, sourcepath::String)
@ Base ./loading.jl:854
[5] _require(pkg::Base.PkgId)
@ Base ./loading.jl:1097
[6] require(uuidkey::Base.PkgId)
@ Base ./loading.jl:1013
[7] require(into::Module, mod::Symbol)
@ Base ./loading.jl:997
during initialization of module GDAL
Ok, thanks. We need a new build of GDAL to support Apple M1. I had already made some preparations to build the new GDAL release (https://github.com/visr/Yggdrasil/tree/gdal34), this should include M1 support. Will try to finish this soon.
No problem. Thank you for the info. I will continue to 7se Julia 1.6.5 for now.
On Thu., Jan. 6, 2022, 07:54 Martijn Visser, @.***> wrote:
Ok, thanks. We need a new build of GDAL to support Apple M1. I had already made some preparations to build the new GDAL release ( https://github.com/visr/Yggdrasil/tree/gdal34), this should include M1 support. Will try to finish this soon.
β Reply to this email directly, view it on GitHub https://github.com/JuliaGeo/GDAL.jl/issues/130#issuecomment-1006565584, or unsubscribe https://github.com/notifications/unsubscribe-auth/AQAWYQNFNFPYR7E5VAJ6M3DUUWGG5ANCNFSM5LLFEYEA . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.
You are receiving this because you authored the thread.Message ID: @.***>
Could you add GDAL#gdal34
and see if the tests pass? I can't test it on Apple M1.
I will check this on an M1 next week. ππ»
Seems to install and load okay, by creates some issues with other packages including ArchGDAL.jl and Proj4.jl. Looks like an older, incompatible version of Proj4.jl or Proj_jll is anchored.
(@v1.7) pkg> add GDAL#gdal34
Cloning git-repo `https://github.com/JuliaGeo/GDAL.jl.git`
Updating git-repo `https://github.com/JuliaGeo/GDAL.jl.git`
Updating registry at `~/.julia/registries/General.toml`
Resolving package versions...
Installed GDAL_jll βββββββ v300.400.100+0
Installed StaticArrays βββ v1.3.1
Installed libgeotiff_jll β v1.7.0+0
Installed CEnum ββββββββββ v0.2.0
Installed PlutoUI ββββββββ v0.7.29
Installed BinaryProvider β v0.5.10
Installed GLM ββββββββββββ v1.6.0
Installed MixedModels ββββ v4.6.0
Installed Proj4 ββββββββββ v0.6.0
Installed TimeZones ββββββ v1.7.1
Installed LLVMExtra_jll ββ v0.0.13+1
Installed GEOS_jll βββββββ v3.10.0+0
Installed PROJ_jll βββββββ v800.200.100+0
Installed CUDA βββββββββββ v3.6.3
Downloaded artifact: GDAL
Downloaded artifact: libgeotiff
Downloaded artifact: LLVMExtra
Downloaded artifact: Ogg
Downloaded artifact: GEOS
Downloaded artifact: PROJ
Updating `~/.julia/environments/v1.7/Project.toml`
[add2ef01] ~ GDAL v1.2.6 β v1.2.6 `https://github.com/JuliaGeo/GDAL.jl.git#gdal34`
[38e38edf] β GLM v1.5.1 β v1.6.0
[ff71e718] β MixedModels v4.5.0 β v4.6.0
[7f904dfe] β PlutoUI v0.7.27 β v0.7.29
[9a7e659c] β Proj4 v0.7.6 β v0.6.0
Updating `~/.julia/environments/v1.7/Manifest.toml`
[79e6a3ab] β Adapt v3.3.2 β v3.3.3
[b99e7846] + BinaryProvider v0.5.10
[fa961155] β CEnum v0.4.1 β v0.2.0
[052768ef] β CUDA v3.6.2 β v3.6.3
[6b39b394] β CodecZstd v0.7.2 β v0.7.0
[150eb455] - CoordinateTransformations v0.6.2
[add2ef01] ~ GDAL v1.2.6 β v1.2.6 `https://github.com/JuliaGeo/GDAL.jl.git#gdal34`
[38e38edf] β GLM v1.5.1 β v1.6.0
[b8f27783] β MathOptInterface v0.10.6 β v0.10.7
[ff71e718] β MixedModels v4.5.0 β v4.6.0
[7f904dfe] β PlutoUI v0.7.27 β v0.7.29
[9a7e659c] β Proj4 v0.7.6 β v0.6.0
[276daf66] β SpecialFunctions v1.8.1 β v2.0.0
[90137ffa] β StaticArrays v1.3.0 β v1.3.1
[f269a46b] β TimeZones v1.7.0 β v1.7.1
[a7073274] β GDAL_jll v300.202.100+0 β v300.400.100+0
[d604d12d] β GEOS_jll v3.9.0+0 β v3.10.0+0
[dad2f222] β LLVMExtra_jll v0.0.13+0 β v0.0.13+1
[e7412a2a] β Ogg_jll v1.3.5+0 β v1.3.5+1
[58948b4f] β PROJ_jll v700.202.100+0 β v800.200.100+0
[06c338fa] β libgeotiff_jll v1.6.0+1 β v1.7.0+0
Building Proj4 βββββ `~/.julia/scratchspaces/44cfe95a-1eb2-52ea-b672-e2afdf69b78f/6a52f9d893b6ad6f15760990ea8adbabd7b64aa7/build.log`
ERROR: Error building `Proj4`:
β Warning: Platform `arm64-apple-darwin21.1.0` is not an officially supported platform
β @ BinaryProvider ~/.julia/packages/BinaryProvider/U2dKK/src/PlatformNames.jl:450
ERROR: LoadError: KeyError: key "unknown" not found
Stacktrace:
[1] getindex
@ ./dict.jl:481 [inlined]
[2] parse_dl_name_version
@ ~/.julia/packages/BinaryProvider/U2dKK/src/PlatformNames.jl:511 [inlined]
[3] detect_libgfortran_abi(libgfortran_name::String, platform::BinaryProvider.UnknownPlatform) (repeats 2 times)
@ BinaryProvider ~/.julia/packages/BinaryProvider/U2dKK/src/PlatformNames.jl:555
[4] detect_libgfortran_abi
@ ~/.julia/packages/BinaryProvider/U2dKK/src/PlatformNames.jl:584 [inlined]
[5] detect_compiler_abi()
@ BinaryProvider ~/.julia/packages/BinaryProvider/U2dKK/src/PlatformNames.jl:667
[6] top-level scope
@ ~/.julia/packages/BinaryProvider/U2dKK/src/PlatformNames.jl:685
[7] include(mod::Module, _path::String)
@ Base ./Base.jl:418
[8] include(x::String)
@ BinaryProvider ~/.julia/packages/BinaryProvider/U2dKK/src/BinaryProvider.jl:1
[9] top-level scope
@ ~/.julia/packages/BinaryProvider/U2dKK/src/BinaryProvider.jl:12
[10] include
@ ./Base.jl:418 [inlined]
[11] include_package_for_output(pkg::Base.PkgId, input::String, depot_path::Vector{String}, dl_load_path::Vector{String}, load_path::Vector{String}, concrete_deps::Vector{Pair{Base.PkgId, UInt64}}, source::String)
@ Base ./loading.jl:1318
[12] top-level scope
@ none:1
[13] eval
@ ./boot.jl:373 [inlined]
[14] eval(x::Expr)
@ Base.MainInclude ./client.jl:453
[15] top-level scope
@ none:1
in expression starting at /Users/stephenwild/.julia/packages/BinaryProvider/U2dKK/src/PlatformNames.jl:685
in expression starting at /Users/stephenwild/.julia/packages/BinaryProvider/U2dKK/src/BinaryProvider.jl:1
ERROR: LoadError: Failed to precompile BinaryProvider [b99e7846-7c00-51b0-8f62-c81ae34c0232] to /Users/stephenwild/.julia/compiled/v1.7/BinaryProvider/jl_o4weR2.
Stacktrace:
[1] error(s::String)
@ Base ./error.jl:33
[2] compilecache(pkg::Base.PkgId, path::String, internal_stderr::IO, internal_stdout::IO, ignore_loaded_modules::Bool)
@ Base ./loading.jl:1466
[3] compilecache(pkg::Base.PkgId, path::String)
@ Base ./loading.jl:1410
[4] _require(pkg::Base.PkgId)
@ Base ./loading.jl:1120
[5] require(uuidkey::Base.PkgId)
@ Base ./loading.jl:1013
[6] require(into::Module, mod::Symbol)
@ Base ./loading.jl:997
[7] include(fname::String)
@ Base.MainInclude ./client.jl:451
[8] top-level scope
@ none:5
in expression starting at /Users/stephenwild/.julia/packages/Proj4/RgJ8T/deps/build.jl:1
(@v1.7) pkg> up Proj4
Updating registry at `~/.julia/registries/General.toml`
No Changes to `~/.julia/environments/v1.7/Project.toml`
No Changes to `~/.julia/environments/v1.7/Manifest.toml`
Precompiling project...
β BinaryProvider
β Proj4
β ArchGDAL
56 dependencies successfully precompiled in 58 seconds (204 already precompiled, 1 skipped during auto due to previous errors)
3 dependencies errored. To see a full report either run `import Pkg; Pkg.precompile()` or load the packages
julia> using GDAL
julia> using ArchGDAL
[ Info: Precompiling ArchGDAL [c9ce4bd3-c3d5-55b8-8973-c0e20141b8c3]
ERROR: LoadError: cfunction: closures are not supported on this platform
Stacktrace:
[1] var"@cplprogress"(__source__::LineNumberNode, __module__::Module, progressfunc::Any)
@ ArchGDAL ~/.julia/packages/ArchGDAL/zkx2f/src/utils.jl:158
[2] #macroexpand#51
@ ./expr.jl:115 [inlined]
[3] macroexpand
@ ./expr.jl:114 [inlined]
[4] docm(source::LineNumberNode, mod::Module, meta::Any, ex::Any, define::Bool) (repeats 2 times)
@ Base.Docs ./docs/Docs.jl:537
[5] (::DocStringExtensions.var"#32#33"{typeof(DocStringExtensions.template_hook)})(::LineNumberNode, ::Vararg{Any})
@ DocStringExtensions ~/.julia/packages/DocStringExtensions/iscC8/src/templates.jl:11
[6] var"@doc"(::LineNumberNode, ::Module, ::String, ::Vararg{Any})
@ Core ./boot.jl:517
[7] include(mod::Module, _path::String)
@ Base ./Base.jl:418
[8] include(x::String)
@ ArchGDAL ~/.julia/packages/ArchGDAL/zkx2f/src/ArchGDAL.jl:1
[9] top-level scope
@ ~/.julia/packages/ArchGDAL/zkx2f/src/ArchGDAL.jl:19
[10] include
@ ./Base.jl:418 [inlined]
[11] include_package_for_output(pkg::Base.PkgId, input::String, depot_path::Vector{String}, dl_load_path::Vector{String}, load_path::Vector{String}, concrete_deps::Vector{Pair{Base.PkgId, UInt64}}, source::Nothing)
@ Base ./loading.jl:1318
[12] top-level scope
@ none:1
[13] eval
@ ./boot.jl:373 [inlined]
[14] eval(x::Expr)
@ Base.MainInclude ./client.jl:453
[15] top-level scope
@ none:1
in expression starting at /Users/stephenwild/.julia/packages/ArchGDAL/zkx2f/src/dataset.jl:1
in expression starting at /Users/stephenwild/.julia/packages/ArchGDAL/zkx2f/src/dataset.jl:1
in expression starting at /Users/stephenwild/.julia/packages/ArchGDAL/zkx2f/src/ArchGDAL.jl:1
ERROR: Failed to precompile ArchGDAL [c9ce4bd3-c3d5-55b8-8973-c0e20141b8c3] to /Users/stephenwild/.julia/compiled/v1.7/ArchGDAL/jl_N1uToA.
Stacktrace:
[1] error(s::String)
@ Base ./error.jl:33
[2] compilecache(pkg::Base.PkgId, path::String, internal_stderr::IO, internal_stdout::IO, ignore_loaded_modules::Bool)
@ Base ./loading.jl:1466
[3] compilecache(pkg::Base.PkgId, path::String)
@ Base ./loading.jl:1410
[4] _require(pkg::Base.PkgId)
@ Base ./loading.jl:1120
[5] require(uuidkey::Base.PkgId)
@ Base ./loading.jl:1013
[6] require(into::Module, mod::Symbol)
@ Base ./loading.jl:997
(@v1.7) pkg> add Proj4@0.7.6
Resolving package versions...
ERROR: Unsatisfiable requirements detected for package Proj4 [9a7e659c]:
Proj4 [9a7e659c] log:
ββpossible versions are: 0.4.0-0.7.6 or uninstalled
ββrestricted to versions 0.7.6 by an explicit requirement, leaving only versions 0.7.6
ββrestricted by compatibility requirements with PROJ_jll [58948b4f] to versions: 0.4.0-0.6.0 or uninstalled β no versions left
ββPROJ_jll [58948b4f] log:
ββpossible versions are: 6.2.1-800.200.100 or uninstalled
ββrestricted to versions 800.200.0-800 by GDAL [add2ef01], leaving only versions 800.200.0-800.200.100
ββGDAL [add2ef01] log:
ββpossible versions are: 1.2.6 or uninstalled
ββGDAL [add2ef01] is fixed to version 1.2.6
Cool, thanks. And if you run test GDAL
? It's right that this branch is not compatible with the latest Proj4.jl in the same environment. There is a PR open in Proj4.jl to make it support the latest binary. Too bad it just jumps down to an old release that had looser compat bounds. But to avoid issues like this it helps not to install everything in the default environment.
And it looks like the ArchGDAL issue is more related to the incomplete support for M1 in Julia proper; it is listed as tier 3 here: https://julialang.org/downloads/#supported_platforms.
Though it's good to be aware of it, and it would be nice if we could get it to work easily.
That's a good point. Here it is a different environment.
The results from test GDAL
(@v1.7) pkg> activate GDALtest
Activating new project at `~/GDALtest`
(GDALtest) pkg> add GDAL#gdal34
Updating git-repo `https://github.com/JuliaGeo/GDAL.jl.git`
Updating registry at `~/.julia/registries/General.toml`
Resolving package versions...
Updating `~/GDALtest/Project.toml`
[add2ef01] + GDAL v1.2.6 `https://github.com/JuliaGeo/GDAL.jl.git#gdal34`
Updating `~/GDALtest/Manifest.toml`
[fa961155] + CEnum v0.4.1
[add2ef01] + GDAL v1.2.6 `https://github.com/JuliaGeo/GDAL.jl.git#gdal34`
[692b3bcd] + JLLWrappers v1.3.0
[21216c6a] + Preferences v1.2.3
[2e619515] + Expat_jll v2.2.10+0
[a7073274] + GDAL_jll v300.400.100+0
[d604d12d] + GEOS_jll v3.10.0+0
[aacddb02] + JpegTurbo_jll v2.1.0+0
[89763e89] + Libtiff_jll v4.3.0+0
[d3a379c0] + LittleCMS_jll v2.12.0+0
[643b3616] + OpenJpeg_jll v2.4.0+0
[58948b4f] + PROJ_jll v800.200.100+0
[76ed43ae] + SQLite_jll v3.37.0+0
[3161d3a3] + Zstd_jll v1.5.0+0
[06c338fa] + libgeotiff_jll v1.7.0+0
[b53b4c65] + libpng_jll v1.6.38+0
[0dad84c5] + ArgTools
[56f22d72] + Artifacts
[2a0f44e3] + Base64
[ade2ca70] + Dates
[f43a241f] + Downloads
[b77e0a4c] + InteractiveUtils
[b27032c2] + LibCURL
[76f85450] + LibGit2
[8f399da3] + Libdl
[56ddb016] + Logging
[d6f4376e] + Markdown
[ca575930] + NetworkOptions
[44cfe95a] + Pkg
[de0858da] + Printf
[3fa0cd96] + REPL
[9a3f8284] + Random
[ea8e919c] + SHA
[9e88b42a] + Serialization
[6462fe0b] + Sockets
[fa267f1f] + TOML
[a4e569a6] + Tar
[cf7118a7] + UUIDs
[4ec0a83e] + Unicode
[deac9b47] + LibCURL_jll
[29816b5a] + LibSSH2_jll
[c8ffd9c3] + MbedTLS_jll
[14a3606d] + MozillaCACerts_jll
[83775a58] + Zlib_jll
[8e850ede] + nghttp2_jll
[3f19e933] + p7zip_jll
(GDALtest) pkg> test GDAL
Testing GDAL
Status `/private/var/folders/r1/t2s9jjkj4k56jy6_29_s3svm0000gn/T/jl_iruozW/Project.toml`
[fa961155] CEnum v0.4.1
[add2ef01] GDAL v1.2.6 `https://github.com/JuliaGeo/GDAL.jl.git#gdal34`
[a7073274] GDAL_jll v300.400.100+0
[58948b4f] PROJ_jll v800.200.100+0
[ca575930] NetworkOptions `@stdlib/NetworkOptions`
[8dfed614] Test `@stdlib/Test`
Status `/private/var/folders/r1/t2s9jjkj4k56jy6_29_s3svm0000gn/T/jl_iruozW/Manifest.toml`
[fa961155] CEnum v0.4.1
[add2ef01] GDAL v1.2.6 `https://github.com/JuliaGeo/GDAL.jl.git#gdal34`
[692b3bcd] JLLWrappers v1.3.0
[21216c6a] Preferences v1.2.3
[2e619515] Expat_jll v2.2.10+0
[a7073274] GDAL_jll v300.400.100+0
[d604d12d] GEOS_jll v3.10.0+0
[aacddb02] JpegTurbo_jll v2.1.0+0
[89763e89] Libtiff_jll v4.3.0+0
[d3a379c0] LittleCMS_jll v2.12.0+0
[643b3616] OpenJpeg_jll v2.4.0+0
[58948b4f] PROJ_jll v800.200.100+0
[76ed43ae] SQLite_jll v3.37.0+0
[3161d3a3] Zstd_jll v1.5.0+0
[06c338fa] libgeotiff_jll v1.7.0+0
[b53b4c65] libpng_jll v1.6.38+0
[0dad84c5] ArgTools `@stdlib/ArgTools`
[56f22d72] Artifacts `@stdlib/Artifacts`
[2a0f44e3] Base64 `@stdlib/Base64`
[ade2ca70] Dates `@stdlib/Dates`
[f43a241f] Downloads `@stdlib/Downloads`
[b77e0a4c] InteractiveUtils `@stdlib/InteractiveUtils`
[b27032c2] LibCURL `@stdlib/LibCURL`
[76f85450] LibGit2 `@stdlib/LibGit2`
[8f399da3] Libdl `@stdlib/Libdl`
[56ddb016] Logging `@stdlib/Logging`
[d6f4376e] Markdown `@stdlib/Markdown`
[ca575930] NetworkOptions `@stdlib/NetworkOptions`
[44cfe95a] Pkg `@stdlib/Pkg`
[de0858da] Printf `@stdlib/Printf`
[3fa0cd96] REPL `@stdlib/REPL`
[9a3f8284] Random `@stdlib/Random`
[ea8e919c] SHA `@stdlib/SHA`
[9e88b42a] Serialization `@stdlib/Serialization`
[6462fe0b] Sockets `@stdlib/Sockets`
[fa267f1f] TOML `@stdlib/TOML`
[a4e569a6] Tar `@stdlib/Tar`
[8dfed614] Test `@stdlib/Test`
[cf7118a7] UUIDs `@stdlib/UUIDs`
[4ec0a83e] Unicode `@stdlib/Unicode`
[deac9b47] LibCURL_jll `@stdlib/LibCURL_jll`
[29816b5a] LibSSH2_jll `@stdlib/LibSSH2_jll`
[c8ffd9c3] MbedTLS_jll `@stdlib/MbedTLS_jll`
[14a3606d] MozillaCACerts_jll `@stdlib/MozillaCACerts_jll`
[83775a58] Zlib_jll `@stdlib/Zlib_jll`
[8e850ede] nghttp2_jll `@stdlib/nghttp2_jll`
[3f19e933] p7zip_jll `@stdlib/p7zip_jll`
Testing Running tests...
β Info: GDAL 3.4.1, released 2021/12/27
β 196 GDAL drivers found
β 69 OGR drivers found
CPLDestroyMutex: Error = 16 (Resource busy)
Test Summary: | Pass Total
GDAL | 129 129
Testing GDAL tests passed
Great, thanks, that means that the GDAL build is working on M1. This issue will be closed automatically when #131 is merged. I'll leave it open for a bit to see if there is feedback.
Can also confirm this is working natively on M1. π
But also that there are some driver regressions
β Info: GDAL 3.2.1, released 2020/12/29
β 204 GDAL drivers found
β 77 OGR drivers found
β Info: GDAL 3.4.1, released 2021/12/27
β 196 GDAL drivers found
β 69 OGR drivers found
Hmm interesting, on Windows I have a few more on GDAL 3.4.1, but also less than GDAL 3.2.1. If there are important ones you care about feel free to add them to https://github.com/JuliaGeo/GDAL.jl/blob/master/test/drivers.jl.
It is true that some drivers are being deprecated / have been removed, e.g. see this entry in the 3.4.1 NEWS:
Remainder: DODS, JPEG2000(Jasper), JPEGLS, MG4LIDAR, FUJIBAS, IDA, INGR and vector driver ARCGEN, ArcObjects, CLOUDANT, COUCHDB, DB2, DODS, FME, GEOMEDIA, GTM, INGRES, MONGODB, REC, WALK are planned for removal in GDAL 3.5. As well as Perl bindings
I am trying to use GDAL via ArchGDAL in Julia 1.7. Unfortunately, I get the following error:
GDAL and ArchGDAL load fine in Julia 1.6.5.
I think this problem rests with GDAL.jl. Any suggestions for how to fix it?