CRASH on MACOS: when opening GeoPackage causes crash in VectorExtractor (SQLite)

clemens-tolboom commented 3 years ago

Steps to reproduce

compile project as described in the README.md
Find the path to Godot which is probably /Applications/Godot.app
Open terminal in this projects directory demo directory

Run from the terminal in demo directory the scene GeopackageDemo.tscn.

/Applications/Godot.app/Contents/MacOS/Godot GeopackageDemo.tscn
/Applications/Godot_mono.app/Contents/MacOS/Godot GeopackageDemo.tscn

Crash Godot on MaxOS Big Sur

This is from Godot_v3.2.4-rc1_osx.universal.app

...
Registered camera FaceTime HD Camera (Display) with id 5 position 0 at index 4
    - 'VirtualXPath'    [XML Path Language - XPath]
handle_crash: Program crashed with signal 11
ERROR: notification: NativeScriptInstance detected crash on method: get_dataset
   At: modules/gdnative/nativescript/nativescript.cpp:749.
Dumping the backtrace. Please include this when reporting the bug on https://github.com/godotengine/godot/issues
[1] 1   libsystem_platform.dylib            0x00007fff203abd7d _sigtramp + 29
[2] 2   ???                                 0x0000000114801080 0x0 + 4638904448

[3] 3   libsqlite3.0.dylib                  0x000000012b3996c7 sqlite3VdbeMemSetStr + 109
[4] 4   libsqlite3.0.dylib                  0x000000012b3e0e27 sqlite3Select + 992
[5] 5   libsqlite3.0.dylib                  0x000000012b3d7f25 yy_reduce + 3218
[6] 6   libsqlite3.0.dylib                  0x000000012b39fc5d sqlite3RunParser + 720
[7] 7   libsqlite3.0.dylib                  0x000000012b3d2c04 sqlite3Prepare + 380
[8] 8   libsqlite3.0.dylib                  0x000000012b39ef9f sqlite3LockAndPrepare + 176
[9] 9   libsqlite3.0.dylib                  0x000000012b39e2e2 sqlite3_exec + 176
[10] 10  libsqlite3.0.dylib                  0x000000012b39f33a sqlite3_get_table + 209

[11] SQLQuery(sqlite3*, char const*, SQLResult*)
[12] GDALGeoPackageDataset::Open(GDALOpenInfo*)
[13] OGRGeoPackageDriverOpen(GDALOpenInfo*)
[14] 14  libgdal.28.dylib                    0x0000000129278f92 GDALOpenEx + 1690
[15] NativeDataset::NativeDataset(char const*)
[16] NativeDataset::NativeDataset(char const*)
[17] VectorExtractor::open_dataset(char const*)
[18] godot::GeoDataset::load_from_file(godot::String)
[19] godot::Geodot::get_dataset(godot::String)
...

kb173 commented 3 years ago

To reproduce, open and play the GeopackageDemo.tscn scene in Godot. (And build Geodot for Mac OS first.)

It would be great if someone else who uses Mac OS could test this! We're not sure whether it's a general problem and what might cause it.

clemens-tolboom commented 3 years ago

Godot_mono.app

=================================================================
    Native stacktrace:
=================================================================
    0x106f866b8 - /Applications/Godot_mono.app/Contents/MacOS/Godot : mono_dump_native_crash_info
    0x106f79f25 - /Applications/Godot_mono.app/Contents/MacOS/Godot : mono_handle_native_crash
    0x106f8db1f - /Applications/Godot_mono.app/Contents/MacOS/Godot : mono_sigsegv_signal_handler_debug
    0x7fff203abd7d - /usr/lib/system/libsystem_platform.dylib : _sigtramp
    0x7fa3ebf5eb00 - Unknown

    0x127716c7d - /usr/local/opt/sqlite/lib/libsqlite3.0.dylib : sqlite3VdbeMemSetStr
    0x12775e549 - /usr/local/opt/sqlite/lib/libsqlite3.0.dylib : sqlite3Select
    0x12775560f - /usr/local/opt/sqlite/lib/libsqlite3.0.dylib : yy_reduce
    0x12771d293 - /usr/local/opt/sqlite/lib/libsqlite3.0.dylib : sqlite3RunParser
    0x1277502ee - /usr/local/opt/sqlite/lib/libsqlite3.0.dylib : sqlite3Prepare
    0x12771c602 - /usr/local/opt/sqlite/lib/libsqlite3.0.dylib : sqlite3LockAndPrepare
    0x12771b960 - /usr/local/opt/sqlite/lib/libsqlite3.0.dylib : sqlite3_exec
    0x12771c987 - /usr/local/opt/sqlite/lib/libsqlite3.0.dylib : sqlite3_get_table

    0x12591819e - /usr/local/opt/gdal/lib/libgdal.28.dylib : _Z8SQLQueryP7sqlite3PKcP9SQLResult
    0x1257c179c - /usr/local/opt/gdal/lib/libgdal.28.dylib : _ZN21GDALGeoPackageDataset4OpenEP12GDALOpenInfo
    0x1257cd7c3 - /usr/local/opt/gdal/lib/libgdal.28.dylib : _ZL23OGRGeoPackageDriverOpenP12GDALOpenInfo
    0x1255aef92 - /usr/local/opt/gdal/lib/libgdal.28.dylib : GDALOpenEx

    0x12505afa3 - /Users/xx/Projects/geodot-plugin/demo/build/libVectorExtractor.dylib : _ZN13NativeDatasetC2EPKc
    0x125058a9d - /Users/xx/Projects/geodot-plugin/demo/build/libVectorExtractor.dylib : _ZN13NativeDatasetC1EPKc
    0x125058a50 - /Users/xx/Projects/geodot-plugin/demo/build/libVectorExtractor.dylib : _ZN15VectorExtractor12open_datasetEPKc
    0x124d6e4d8 - /Users/xx/Projects/geodot-plugin/demo/addons/geodot/osx/libgeodot.dylib : _ZN5godot10GeoDataset14load_from_fileENS_6StringE
    0x124d72548 - /Users/xx/Projects/geodot-plugin/demo/addons/geodot/osx/libgeodot.dylib : _ZN5godot6Geodot11get_datasetENS_6StringE
    0x124d72c71 - /Users/xx/Projects/geodot-plugin/demo/addons/geodot/osx/libgeodot.dylib : _ZN5godot14_WrappedMethodINS_6GeodotENS_3RefINS_10GeoDatasetEEEJNS_6StringEEE5applyIJLi0EEEEvPNS_7VariantEPS1_PS9_NS_10__SequenceIJXspT_EEEE
    0x124d72bf8 - /Users/xx/Projects/geodot-plugin/demo/addons/geodot/osx/libgeodot.dylib : _ZN5godot16__wrapped_methodINS_6GeodotENS_3RefINS_10GeoDatasetEEEJNS_6StringEEEE13godot_variantPvS7_S7_iPPS6_

    0x1074dfaf1 - /Applications/Godot_mono.app/Contents/MacOS/Godot : _ZN20NativeScriptInstance4callERK10StringNamePPK7VariantiRNS3_9CallErrorE
    0x109996120 - /Applications/Godot_mono.app/Contents/MacOS/Godot : _ZN6Object4callERK10StringNamePPK7VariantiRNS3_9CallErrorE
    0x109a5ec0f - /Applications/Godot_mono.app/Contents/MacOS/Godot : _ZN7Variant8call_ptrERK10StringNamePPKS_iPS_RNS_9CallErrorE
    0x10758ecf9 - /Applications/Godot_mono.app/Contents/MacOS/Godot : _ZN16GDScriptFunction4callEP16GDScriptInstancePPK7VariantiRNS2_9CallErrorEPNS_9CallStateE
    0x107520ea7 - /Applications/Godot_mono.app/Contents/MacOS/Godot : _ZN16GDScriptInstance17_ml_call_reversedEP8GDScriptRK10StringNamePPK7Varianti
    0x109992c93 - /Applications/Godot_mono.app/Contents/MacOS/Godot : _ZN6Object12notificationEib
    0x10897cefb - /Applications/Godot_mono.app/Contents/MacOS/Godot : _ZN4Node16_propagate_readyEv
    0x10897cebb - /Applications/Godot_mono.app/Contents/MacOS/Godot : _ZN4Node16_propagate_readyEv
    0x10898276f - /Applications/Godot_mono.app/Contents/MacOS/Godot : _ZN4Node9_set_treeEP9SceneTree
    0x1089b27cf - /Applications/Godot_mono.app/Contents/MacOS/Godot : _ZN9SceneTree4initEv
    0x1071e6264 - /Applications/Godot_mono.app/Contents/MacOS/Godot : _ZN6OS_OSX3runEv
    0x1071ec451 - /Applications/Godot_mono.app/Contents/MacOS/Godot : main
    0x7fff20382621 - /usr/lib/system/libdyld.dylib : start
    0xa - Unknown

clemens-tolboom commented 3 years ago

geodot.gdnlib

Was missing

[dependencies]
...
OSX.64=[ "res://build/libRasterTileExtractor.dylib", "res://build/libRasterTileExtractor.dylib", "/usr/local/opt/gdal/lib/libgdal.dylib" ]

I thought maybe adding libgdal could help the build but that failed.

OSX.64=[ "res://build/libRasterTileExtractor.dylib", "res://build/libRasterTileExtractor.dylib", "/usr/local/opt/gdal/lib/libgdal.dylib" ]

Velek commented 3 years ago

I always keep my systems updated, so they’re on the latest Big Sur point release.

clemens-tolboom commented 3 years ago

Someone suggested to log SQLite queries ... is that possible?

kb173 commented 3 years ago

Unfortunately I couldn't find any option for that :/ I did find a way to enable slightly more verbose logging though.

I created a very minimal reproduction project for this bug: geodot-mac-test.zip @clemens-tolboom could you try compiling and running this? (scons platform=osx osgeo_path=...; build/gpkg_test)

While creating this, I noticed that I also get a segfault if I don't call GDALAllRegister();. This is called in Geodot (in VectorExtractor::initialize(), which is called by Geodot::_init()), but if this project does not crash for you, maybe Geodot::_init() isn't correctly called on OSX for some reason? If it does crash, we could use this minimal project and maybe talk to someone at GDAL about it.

Thanks and I'm looking forward to hearing about the results!

clemens-tolboom commented 3 years ago

Some comments moved to #57

clemens-tolboom commented 3 years ago

Testing the minimal project

scons target=release platform=osx osgeo_path=/usr/local/Cellar/gdal/3.2.1/

./build/gpkg_test 
Opening dataset at geopackage_sample.gpkg
    - 'VirtualXPath'    [XML Path Language - XPath]
GPKG: GeoPackage v1.2.0
GDAL: GDALOpen(geopackage_sample.gpkg, this=0x7fbbf2836200) succeeds as GPKG.
Opened.

Dataset has ...
1 layers
Name of first layer:
geopackage_sample

clemens-tolboom commented 3 years ago

scons target=release platform=osx osgeo_path=/usr/local/Cellar/gdal/3.2.1/
scons: Reading SConscript files ...
scons: done reading SConscript files.
scons: Building targets ...
g++ -o main.o -c -std=c++17 -I/usr/local/Cellar/gdal/3.2.1/include main.cpp
g++ -o build/gpkg_test main.o -L/usr/local/Cellar/gdal/3.2.1/lib -lgdal
scons: done building targets.

Commenting out //GDALAllRegister(); gives

./build/gpkg_test 
Opening dataset at geopackage_sample.gpkg
Opened.

Dataset has ...
Segmentation fault: 11

boku-ilen / geodot-plugin