ExecuteReader (sqlite3_prepare) Fail: SIGSEGV

AceCoderLaura commented 5 years ago

Steps to Reproduce

Invoke Mono.Data.Sqlite.SqliteCommand.ExecuteReader()

Expected Behavior

Executes the reader.

Actual Behavior

Get a SIGSEGV while executing native code.

Environment

JetBrains Rider 2018.3.4
Build #RD-183.6092.12, built on March 4, 2019
Licensed to Laura Bailey
Subscription is active until August 13, 2019
JRE: 1.8.0_152-release-1343-b28 amd64
JVM: OpenJDK 64-Bit Server VM by JetBrains s.r.o
Windows 10 10.0

=== Visual Studio Community 2017 for Mac ===

Version 7.8.1 (build 4)
Installation UUID: 0fd06d7a-7f21-40f4-890c-9189ea7bfc77
    GTK+ 2.24.23 (Raleigh theme)
    Xamarin.Mac 5.0.0.0 ( / b40230c0)

    Package version: 516000221

=== Mono Framework MDK ===

Runtime:
    Mono 5.16.0.221 (2018-06/b63e5378e38) (64-bit)
    Package version: 516000221

=== NuGet ===

Version: 4.8.0.5385

=== .NET Core ===

Runtime: /usr/local/share/dotnet/dotnet
Runtime Versions:
    2.1.8
    2.1.7
SDK: /usr/local/share/dotnet/sdk/2.1.504/Sdks
SDK Versions:
    2.1.504
    2.1.503
MSBuild SDKs: /Library/Frameworks/Mono.framework/Versions/5.16.0/lib/mono/msbuild/15.0/bin/Sdks

=== Xamarin.Profiler ===

Version: 1.6.4
Location: /Applications/Xamarin Profiler.app/Contents/MacOS/Xamarin Profiler

=== Updater ===

Version: 11

=== Apple Developer Tools ===

Xcode 10.1 (14460.46)
Build 10B61

=== Xamarin.Mac ===

Version: 5.2.1.15 (Visual Studio Community)
Hash: d60abd198
Branch: 
Build date: 2019-02-01 12:23:30-0500

=== Xamarin.iOS ===

Version: 12.2.1.15 (Visual Studio Community)
Hash: d60abd198
Branch: d15-9
Build date: 2019-02-01 12:23:29-0500

=== Xamarin.Android ===

Not Installed

=== Microsoft Mobile OpenJDK ===

Java SDK: Not Found

Android Designer EPL code available here:
https://github.com/xamarin/AndroidDesigner.EPL

=== Android Device Manager ===

Version: 7.8.1.0
Hash: 8924ea4a

=== Xamarin Inspector ===

Version: 1.4.3
Hash: db27525
Branch: 1.4-release
Build date: Mon, 09 Jul 2018 21:20:18 GMT
Client compatibility: 1

=== Build Information ===

Release ID: 708010004
Git revision: 8e2582c0fc1558364b67502b43d2e6a44ff6bc48
Build date: 2019-02-21 18:39:01+00
Build branch: release-7.8
Xamarin extensions: af96e69ddf566b275db40628f8530ca48a97f0b9

=== Operating System ===

Mac OS X 10.14.3
Darwin 18.2.0 Darwin Kernel Version 18.2.0
    Thu Dec 20 20:46:53 PST 2018
    root:xnu-4903.241.1~1/RELEASE_X86_64 x86_64

Build Logs

JetBrainsLog.ReSharperBuild_2019_03_20_05_47_07_6136.log.zip

Stack Trace

2019-03-20 16:48:12.635860+1100 Adapt.Presentation.Xivic.iOS[7868:101522] critical: Stacktrace:
2019-03-20 16:48:12.636009+1100 Adapt.Presentation.Xivic.iOS[7868:101522] critical:   at <unknown> <0xffffffff>
2019-03-20 16:48:12.636143+1100 Adapt.Presentation.Xivic.iOS[7868:101522] critical:   at (wrapper managed-to-native) Mono.Data.Sqlite.UnsafeNativeMethods.sqlite3_prepare (intptr,intptr,int,intptr&,intptr&) [0x0000e] in <aee62de483844680824c6e8a854b2636>:0
2019-03-20 16:48:12.636249+1100 Adapt.Presentation.Xivic.iOS[7868:101522] critical:   at Mono.Data.Sqlite.SQLite3.Prepare (Mono.Data.Sqlite.SqliteConnection,string,Mono.Data.Sqlite.SqliteStatement,uint,string&) [0x00044] in /Library/Frameworks/Xamarin.iOS.framework/Versions/12.2.1.15/src/Xamarin.iOS/mcs/class/Mono.Data.Sqlite/Mono.Data.Sqlite_2.0/SQLite3.cs:268
2019-03-20 16:48:12.636322+1100 Adapt.Presentation.Xivic.iOS[7868:101522] critical:   at Mono.Data.Sqlite.SqliteCommand.BuildNextCommand () [0x00016] in /Library/Frameworks/Xamarin.iOS.framework/Versions/12.2.1.15/src/Xamarin.iOS/mcs/class/Mono.Data.Sqlite/Mono.Data.Sqlite_2.0/SQLiteCommand.cs:230
2019-03-20 16:48:12.636414+1100 Adapt.Presentation.Xivic.iOS[7868:101522] critical:   at Mono.Data.Sqlite.SqliteCommand.GetStatement (int) [0x00008] in /Library/Frameworks/Xamarin.iOS.framework/Versions/12.2.1.15/src/Xamarin.iOS/mcs/class/Mono.Data.Sqlite/Mono.Data.Sqlite_2.0/SQLiteCommand.cs:264
2019-03-20 16:48:12.636567+1100 Adapt.Presentation.Xivic.iOS[7868:101522] critical:   at Mono.Data.Sqlite.SqliteDataReader.NextResult () [0x000b3] in /Library/Frameworks/Xamarin.iOS.framework/Versions/12.2.1.15/src/Xamarin.iOS/mcs/class/Mono.Data.Sqlite/Mono.Data.Sqlite_2.0/SQLiteDataReader.cs:916
2019-03-20 16:48:12.636657+1100 Adapt.Presentation.Xivic.iOS[7868:101522] critical:   at Mono.Data.Sqlite.SqliteDataReader..ctor (Mono.Data.Sqlite.SqliteCommand,System.Data.CommandBehavior) [0x0004e] in /Library/Frameworks/Xamarin.iOS.framework/Versions/12.2.1.15/src/Xamarin.iOS/mcs/class/Mono.Data.Sqlite/Mono.Data.Sqlite_2.0/SQLiteDataReader.cs:89
2019-03-20 16:48:12.636763+1100 Adapt.Presentation.Xivic.iOS[7868:101522] critical:   at Mono.Data.Sqlite.SqliteCommand.ExecuteReader (System.Data.CommandBehavior) [0x00006] in /Library/Frameworks/Xamarin.iOS.framework/Versions/12.2.1.15/src/Xamarin.iOS/mcs/class/Mono.Data.Sqlite/Mono.Data.Sqlite_2.0/SQLiteCommand.cs:539
2019-03-20 16:48:12.636849+1100 Adapt.Presentation.Xivic.iOS[7868:101522] critical:   at Mono.Data.Sqlite.SqliteCommand.ExecuteReader () [0x00000] in /Library/Frameworks/Xamarin.iOS.framework/Versions/12.2.1.15/src/Xamarin.iOS/mcs/class/Mono.Data.Sqlite/Mono.Data.Sqlite_2.0/SQLiteCommand.cs:551
2019-03-20 16:48:12.636936+1100 Adapt.Presentation.Xivic.iOS[7868:101522] critical:   at Adapt.Data.MonoSQLite.MonoSQLiteCommand.ExecuteReader () [0x00000] in C:\AdaptSource\src\Adapt.Xamarin.Shared\SQLiteInjectable.cs:110
2019-03-20 16:48:12.637031+1100 Adapt.Presentation.Xivic.iOS[7868:101522] critical:   at Adapt.Data.MonoSQLite.MonoSQLiteDataAdapter.Fill (Adapt.Data.DataTable) [0x0002c] in C:\AdaptSource\src\Adapt.Xamarin.Shared\SQLiteInjectable.cs:43
2019-03-20 16:48:12.637133+1100 Adapt.Presentation.Xivic.iOS[7868:101522] critical:   at Adapt.Data.Generic.SQLite.SQLiteDatabaseAdapter.Fill (Adapt.Data.IDbDataAdapter,Adapt.Data.DataTable) [0x00000] in C:\AdaptSource\src\Xivic\Adapt.Data.Generic.Standard\Adapt\Data\Generic\ADO\SQLite\SQLiteDatabaseAdapter.cs:254
2019-03-20 16:48:12.637336+1100 Adapt.Presentation.Xivic.iOS[7868:101522] critical:   at Adapt.Data.Generic.ADO.ADOBasedDatabaseAdapterBase.Fill (string,System.Collections.ObjectModel.Collection`1<Adapt.Data.Generic.DataParameter>,Adapt.Model.Data.ConnectionInfo,Adapt.Model.Data.TableInfo) [0x0001a] in C:\AdaptSource\src\Xivic\Adapt.Data.Generic\Adapt\Data\Generic\ADO\ADOBasedDatabaseAdapterBase.cs:143
2019-03-20 16:48:12.637419+1100 Adapt.Presentation.Xivic.iOS[7868:101522] critical:   at Adapt.Data.Generic.SQLite.SQLiteDataSchemaAdapter.GetDatabaseSchema (Adapt.Model.Data.ConnectionInfo,string,string) [0x0008a] in C:\AdaptSource\src\Xivic\Adapt.Data.Generic.Standard\Adapt\Data\Generic\ADO\SQLite\SQLiteDataSchemaAdapter.cs:46
2019-03-20 16:48:12.637547+1100 Adapt.Presentation.Xivic.iOS[7868:101522] critical:   at Adapt.Data.Generic.DataSchemaAdapterBase.GenerateUpdateStatements (System.Collections.ObjectModel.Collection`1<Adapt.Model.DataSchema.DSColumn>,System.Collections.ObjectModel.Collection`1<Adapt.Model.DataSchema.DSKey>,System.Collections.ObjectModel.Collection`1<Adapt.Model.DataSchema.DSIndexColumn>,Adapt.Model.Data.ConnectionInfo) [0x00001] in C:\AdaptSource\src\Xivic\Adapt.Data.Generic\Adapt\Data\Generic\DataSchemaAdapterBase.cs:31
2019-03-20 16:48:12.637645+1100 Adapt.Presentation.Xivic.iOS[7868:101522] critical:   at Adapt.Data.Generic.DataAccessLayer.GenerateUpdateStatements (System.Collections.ObjectModel.Collection`1<Adapt.Model.DataSchema.DSColumn>,System.Collections.ObjectModel.Collection`1<Adapt.Model.DataSchema.DSKey>,System.Collections.ObjectModel.Collection`1<Adapt.Model.DataSchema.DSIndexColumn>,Adapt.Model.Data.ConnectionInfo) [0x00001] in C:\AdaptSource\src\Xivic\Adapt.Data.Generic\Adapt\Data\Generic\DataAccessLayer.cs:575
2019-03-20 16:48:12.637735+1100 Adapt.Presentation.Xivic.iOS[7868:101522] critical:   at Adapt.XivicClient.Sync.SyncInitializationClient/<>c__DisplayClass25_0.<CreateDatabaseAsync>b__1 (Adapt.Model.Data.ConnectionInfo) [0x00001] in C:\AdaptSource\src\Xivic\Adapt.XivicClient.Database.Standard\Adapt\XivicClient\Sync\SyncInitializationClient.cs:205
2019-03-20 16:48:12.637817+1100 Adapt.Presentation.Xivic.iOS[7868:101522] critical:   at Adapt.Data.Generic.TransactionWrapper.Wrap (Adapt.Data.Generic.TransactionWrapDelegate,Adapt.Model.Data.ConnectionInfo&) [0x00089] in C:\AdaptSource\src\Xivic\Adapt.Data.Generic\Adapt\Data\Generic\TransactionWrapper.cs:59
2019-03-20 16:48:12.637892+1100 Adapt.Presentation.Xivic.iOS[7868:101522] critical:   at Adapt.XivicClient.Sync.SyncInitializationClient/<>c__DisplayClass25_0.<CreateDatabaseAsync>b__0 () [0x00001] in C:\AdaptSource\src\Xivic\Adapt.XivicClient.Database.Standard\Adapt\XivicClient\Sync\SyncInitializationClient.cs:202
2019-03-20 16:48:12.637987+1100 Adapt.Presentation.Xivic.iOS[7868:101522] critical:   at System.Threading.Tasks.Task.InnerInvoke () [0x0000f] in /Library/Frameworks/Xamarin.iOS.framework/Versions/12.2.1.15/src/Xamarin.iOS/external/corert/src/System.Private.CoreLib/src/System/Threading/Tasks/Task.cs:2501
2019-03-20 16:48:12.638081+1100 Adapt.Presentation.Xivic.iOS[7868:101522] critical:   at System.Threading.Tasks.Task.Execute () [0x00000] in /Library/Frameworks/Xamarin.iOS.framework/Versions/12.2.1.15/src/Xamarin.iOS/external/corert/src/System.Private.CoreLib/src/System/Threading/Tasks/Task.cs:2344
2019-03-20 16:48:12.638191+1100 Adapt.Presentation.Xivic.iOS[7868:101522] critical:   at System.Threading.Tasks.Task.ExecutionContextCallback (object) [0x00006] in /Library/Frameworks/Xamarin.iOS.framework/Versions/12.2.1.15/src/Xamarin.iOS/external/corert/src/System.Private.CoreLib/src/System/Threading/Tasks/Task.cs:2488
2019-03-20 16:48:12.638278+1100 Adapt.Presentation.Xivic.iOS[7868:101522] critical:   at System.Threading.ExecutionContext.RunInternal (System.Threading.ExecutionContext,System.Threading.ContextCallback,object,bool) [0x00071] in /Library/Frameworks/Xamarin.iOS.framework/Versions/12.2.1.15/src/Xamarin.iOS/mcs/class/referencesource/mscorlib/system/threading/executioncontext.cs:961
2019-03-20 16:48:12.638349+1100 Adapt.Presentation.Xivic.iOS[7868:101522] critical:   at System.Threading.ExecutionContext.Run (System.Threading.ExecutionContext,System.Threading.ContextCallback,object,bool) [0x00000] in /Library/Frameworks/Xamarin.iOS.framework/Versions/12.2.1.15/src/Xamarin.iOS/mcs/class/referencesource/mscorlib/system/threading/executioncontext.cs:908
2019-03-20 16:48:12.638417+1100 Adapt.Presentation.Xivic.iOS[7868:101522] critical:   at System.Threading.Tasks.Task.ExecuteWithThreadLocal (System.Threading.Tasks.Task&) [0x00034] in /Library/Frameworks/Xamarin.iOS.framework/Versions/12.2.1.15/src/Xamarin.iOS/mcs/class/corlib/corert/Task.cs:72
2019-03-20 16:48:12.638523+1100 Adapt.Presentation.Xivic.iOS[7868:101522] critical:   at System.Threading.Tasks.Task.ExecuteEntry (bool) [0x0004a] in /Library/Frameworks/Xamarin.iOS.framework/Versions/12.2.1.15/src/Xamarin.iOS/external/corert/src/System.Private.CoreLib/src/System/Threading/Tasks/Task.cs:2410
2019-03-20 16:48:12.638613+1100 Adapt.Presentation.Xivic.iOS[7868:101522] critical:   at System.Threading.Tasks.Task.System.Threading.IThreadPoolWorkItem.ExecuteWorkItem () [0x00000] in /Library/Frameworks/Xamarin.iOS.framework/Versions/12.2.1.15/src/Xamarin.iOS/external/corert/src/System.Private.CoreLib/src/System/Threading/Tasks/Task.cs:2359
2019-03-20 16:48:12.638692+1100 Adapt.Presentation.Xivic.iOS[7868:101522] critical:   at System.Threading.ThreadPoolWorkQueue.Dispatch () [0x00074] in /Library/Frameworks/Xamarin.iOS.framework/Versions/12.2.1.15/src/Xamarin.iOS/mcs/class/referencesource/mscorlib/system/threading/threadpool.cs:858
2019-03-20 16:48:12.638765+1100 Adapt.Presentation.Xivic.iOS[7868:101522] critical:   at ObjCRuntime.Runtime.ThreadPoolDispatcher (System.Func`1<bool>) [0x00006] in /Library/Frameworks/Xamarin.iOS.framework/Versions/12.2.1.15/src/Xamarin.iOS/ObjCRuntime/Runtime.cs:280
2019-03-20 16:48:12.638852+1100 Adapt.Presentation.Xivic.iOS[7868:101522] critical:   at System.Threading._ThreadPoolWaitCallback.PerformWaitCallback () [0x00009] in /Library/Frameworks/Xamarin.iOS.framework/Versions/12.2.1.15/src/Xamarin.iOS/mcs/class/referencesource/mscorlib/system/threading/threadpool.cs:1210
2019-03-20 16:48:12.638961+1100 Adapt.Presentation.Xivic.iOS[7868:101522] critical:   at (wrapper runtime-invoke) <Module>.runtime_invoke_bool (object,intptr,intptr,intptr) [0x0001f] in <2b439461a53d406abf16d6e5e34ece7f>:0
2019-03-20 16:48:12.639036+1100 Adapt.Presentation.Xivic.iOS[7868:101522] critical: 
Native stacktrace:
2019-03-20 16:48:12.651049+1100 Adapt.Presentation.Xivic.iOS[7868:101522] critical:     0   Adapt.Presentation.Xivic.iOS        0x000000010267a9f4 mono_handle_native_crash + 244
2019-03-20 16:48:12.651223+1100 Adapt.Presentation.Xivic.iOS[7868:101522] critical:     1   Adapt.Presentation.Xivic.iOS        0x000000010268b310 mono_sigsegv_signal_handler + 288
2019-03-20 16:48:12.651313+1100 Adapt.Presentation.Xivic.iOS[7868:101522] critical:     2   libsystem_platform.dylib            0x000000011827ab3d _sigtramp + 29
2019-03-20 16:48:12.651404+1100 Adapt.Presentation.Xivic.iOS[7868:101522] critical:     3   ???                                 0x0000000002000891 0x0 + 33556625
2019-03-20 16:48:12.651493+1100 Adapt.Presentation.Xivic.iOS[7868:101522] critical:     4   libsqlite3.dylib                    0x00000001168f14b1 sqlite3VdbeMultiLoad + 529
2019-03-20 16:48:12.651577+1100 Adapt.Presentation.Xivic.iOS[7868:101522] critical:     5   libsqlite3.dylib                    0x00000001168ab448 sqlite3Pragma + 14632
2019-03-20 16:48:12.651646+1100 Adapt.Presentation.Xivic.iOS[7868:101522] critical:     6   libsqlite3.dylib                    0x000000011687d265 yy_reduce + 9813
2019-03-20 16:48:12.651721+1100 Adapt.Presentation.Xivic.iOS[7868:101522] critical:     7   libsqlite3.dylib                    0x0000000116812727 sqlite3RunParser + 551
2019-03-20 16:48:12.651788+1100 Adapt.Presentation.Xivic.iOS[7868:101522] critical:     8   libsqlite3.dylib                    0x00000001168107bd sqlite3LockAndPrepare + 2029
2019-03-20 16:48:12.651876+1100 Adapt.Presentation.Xivic.iOS[7868:101522] critical:     9   libsqlite3.dylib                    0x000000011680ffbc sqlite3_prepare + 28
2019-03-20 16:48:12.651940+1100 Adapt.Presentation.Xivic.iOS[7868:101522] critical:     10  ???                                 0x000000013cf7fef8 0x0 + 5317852920
2019-03-20 16:48:12.652024+1100 Adapt.Presentation.Xivic.iOS[7868:101522] critical: 
=================================================================
Got a SIGSEGV while executing native code. This usually indicates
a fatal error in the mono runtime or one of the native libraries 
used by your application.
=================================================================

PS: I get these native errors all the time. How do I break when one is thrown? It would really help me write better issue submissions.

spouliot commented 5 years ago

This is a native crash, not an exception, so nothing is thrown. You can attach lldb to the process but it's a bit harder (different debugger and native code) and it's not your code.

Your best bet is to put a breakpoint on the last managed frame

2019-03-20 16:48:12.636936+1100 Adapt.Presentation.Xivic.iOS[7868:101522] critical:   at Adapt.Data.MonoSQLite.MonoSQLiteCommand.ExecuteReader () [0x00000] in C:\AdaptSource\src\Adapt.Xamarin.Shared\SQLiteInjectable.cs:110

and see the state of your objects. Something (the input) is causing sqlite to crash. It's might be the query (being prepared) or it might be related to how the session was configured (i.e. earlier).

The symbolicated crash report would also help. It will give more some ideas of what the other threads are doing, e.g. do you call sqlite from multiple thread? did you configure your session as such ?

Finally it's possible that sqlite would have written something into the device logs before crashing, e.g. an assert. Reading the logs might include the clue why the application crashed.

AceCoderLaura commented 5 years ago

Alright I guess I'll dig a bit more in the managed code for answers. I was not aware of device logs and symbolic crash reports until now, I'll look into those next and forward them here if there might be anything helpful.

AceCoderLaura commented 5 years ago

The last command to go in before the native library crashes is pragma table_info('TableSchema.TableName'). Perhaps the table does not exist? Sometimes this call succeeds sometimes it doesn't.

The device log doesn't appear to contain any new information other than the SIGSEGV message I already have. I can't get to the symbolic crash report since the simulator appears to hide the menu items in the settings that allow you to get access to them.

AceCoderLaura commented 5 years ago

Okay I've found that there is no specific call that causes this problem. It appears to be completely random.

AceCoderLaura commented 5 years ago

I can have one call succeed and another fail on the same thread. But I can also have multiple calls succeed on different threads. It seems like the thread the statement executes on is irrelevant.

AceCoderLaura commented 5 years ago

I attached LLDB and found the exception EXC_BAD_ACCESS (code=1, address=0x1b00000025). I also retrieved the stack trace:

sqlite3ExprListAppend 0x000000011fa5a9f5
yy_reduce 0x000000011fa3da54
sqlite3RunParser 0x000000011f9d4727
sqlite3LockAndPrepare 0x000000011f9d27bd
sqlite3_prepare 0x000000011f9d1fbc

spouliot commented 5 years ago

I had no clue but it did look like a threading issue. Your additional details makes me even more convinced of it.

WRT threading how did you initialize sqlite ? by default it's not configured for multithreading and you'll need something like this in your code:

SqliteConnection.SetConfig(SQLiteConfig.Serialized);

This is the safest option to use since it (sqlite) will deal with any threading issue.

The simulator does produce crash reports (see Console.app) but they are not very useful (the JIT is used). You'll get a better (more useful one) one when running your application on a device;
From lldb you want to get a backtrace of all threads (not just the current one).

AceCoderLaura commented 5 years ago

It appears that calling Mono.Data.Sqlite.SqliteConnection.SetConfig(Mono.Data.Sqlite.SQLiteConfig.Serialized); fixes the issue, thanks.

I wish the library had told me the problem from the get-go 🙁

spouliot commented 5 years ago

Apple builds/ships libsqlite in the OS as a multithreaded by default (i.e. it puts your in charge). However this library has a mode that can take care of multithreading (by serializing requests) for you.

In general you are responsible to protect the state of your data when using multi-threading. Several types are not thread-safe, limited to the main thread (mostly UI) or to the thread that created them.

xamarin / xamarin-macios