search

groue / GRDB.swift

A toolkit for SQLite databases, with a focus on application development
MIT License
6.91k stars 711 forks source link

EXC_BAD_ACCESS: Exception 1, Code 1868215664, Subcode 8 > Stack overflow in __cxa_throw #1209

Closed xmanu closed 2 years ago

xmanu commented 2 years ago

What did you do?

We recently released a new app using GRDB for the first time. We are really happy with GRDB so far. Unfortunately we get weird stack overflow errors in our bug tracker that seem to contain no code from our codebase, but seem to lie solely in the GRDB source.

What did you expect to happen?

The app should not crash.

What happened instead?

The app crashes with EXC_BAD_ACCESS: Exception 1, Code 1868215664, Subcode 8 > Stack overflow in __cxa_throw.

Environment

GRDB flavor(s): GRDB GRDB version: 5.19.0 Installation method: SPM Xcode version: 13.3.1 Swift version: 5.6 Platform(s) running GRDB: iOS macOS version running Xcode: 12.3.1

I unfortunately cannot provide a simplified demo project as we cannot even reproduce the bug locally.

I can provide this stacktrace:

OS Version: iOS 15.4.1 (19E258)
Report Version: 104

Exception Type: EXC_BAD_ACCESS (SIGBUS)
Exception Codes: BUS_NOOP at 0x000000016f5ab570
Crashed Thread: 10

Application Specific Information:
Exception 1, Code 1868215664, Subcode 8 >
Stack overflow in __cxa_throw

Thread 10 Crashed:
0   <MyApp>                    0x200ff431c         SQLExpression.sql (SQLExpression.swift:871)
1   <MyApp>                    0x200ff5238         SQLExpression.sql (SQLExpression.swift:997)
2   <MyApp>                    0x20103728c         SQLQualifiedJoin.sql (SQLQueryGenerator.swift:780)
3   <MyApp>                    0x20102ab84         [inlined] sql (SQLQueryGenerator.swift:733)
4   <MyApp>                    0x20102ab84         SQLQueryGenerator.requestSQL (SQLQueryGenerator.swift:45)
5   <MyApp>                    0x201029c04         SQLQueryGenerator.makeStatement (SQLQueryGenerator.swift:112)
6   <MyApp>                    0x201028858         SQLQueryGenerator.makePreparedRequest (SQLQueryGenerator.swift:96)
7   <MyApp>                    0x200fdb980         QueryInterfaceRequest.makePreparedRequest (QueryInterfaceRequest.swift:52)
8   <MyApp>                    0x200fdf200         QueryInterfaceRequest<T>
9   <MyApp>                    0x2010732c8         FetchableRecord.fetchOne<T> (FetchableRecord.swift:406)
10  <MyApp>                    0x200eb65c4         FetchRequest.observeOne<T>
11  <MyApp>                    0x200eb76b4         FetchRequest.observeOne<T>
12  <MyApp>                    0x2010a7a78         ValueReducers.Fetch._fetch (Fetch.swift:14)
13  <MyApp>                    0x2010a7b14         ValueReducers.Fetch<T>
14  <MyApp>                    0x2010a68dc         ValueObserver.fetchUpdatingObserverRegionIfNeeded (ValueObserver.swift:224)
15  <MyApp>                    0x2010a7260         ValueObserver.fetchUpdatingObserverRegionIfNeeded
16  <MyApp>                    0x200f2c6cc         Database.recordingSelection<T> (Database.swift:615)
17  <MyApp>                    0x2010a6c18         ValueObserver.recordingSelectedRegion<T> (ValueObserver.swift:280)
18  <MyApp>                    0x2010a65f0         ValueObserver.updatingObserverRegionIfNeeded<T>
19  <MyApp>                    0x2010a723c         [inlined] fetchUpdatingObserverRegionIfNeeded (ValueObserver.swift:222)
20  <MyApp>                    0x2010a723c         [inlined] ? (ValueObserver.swift:205)
21  <MyApp>                    0x2010a723c         ValueObserver.fetchFutureSync
22  <MyApp>                    0x20109b418         throwingFirstError<T> (Utils.swift:122)
23  <MyApp>                    0x200f2e3c0         [inlined] readOnly (Database.swift:547)
24  <MyApp>                    0x200f2e3c0         Database.isolated<T> (Database.swift:1018)
25  <MyApp>                    0x200f3294c         Database.isolated<T>
26  <MyApp>                    0x200f2dc28         Database.inTransaction (Database.swift:975)
27  <MyApp>                    0x200f2e558         Database.inSavepoint (Database.swift:1066)
28  <MyApp>                    0x200f2e280         Database.isolated<T> (Database.swift:1016)
29  <MyApp>                    0x2010a6774         [inlined] ? (ValueObserver.swift:204)
30  <MyApp>                    0x2010a6774         ValueObserver.fetchFutureSync (ValueObserver.swift:203)
31  <MyApp>                    0x2010a619c         [inlined] fetchFuture
32  <MyApp>                    0x2010a619c         ValueObserver.databaseDidCommit (ValueObserver.swift:100)
33  <MyApp>                    0x200fb7f9c         [inlined] databaseDidCommit (TransactionObserver.swift:850)
34  <MyApp>                    0x200fb7f9c         DatabaseObservationBroker.databaseDidCommit (TransactionObserver.swift:447)
35  <MyApp>                    0x200fb6c98         [inlined] databaseDidCommitEmptyDeferredTransaction (TransactionObserver.swift:490)
36  <MyApp>                    0x200fb6c98         DatabaseObservationBroker.statementDidExecute (TransactionObserver.swift:346)
37  <MyApp>                    0x200f9c46c         [inlined] statementDidExecute (Database+Statements.swift:570)
38  <MyApp>                    0x200f9c46c         Statement.execute (Statement.swift:304)
39  <MyApp>                    0x200f2c218         Database.withAuthorizer<T> (<compiler-generated>:592)
40  <MyApp>                    0x200f246fc         [inlined] execute (Statement.swift:299)
41  <MyApp>                    0x200f246fc         Database.execute (Database+Statements.swift:465)
42  <MyApp>                    0x200f2dd74         [inlined] execute (Database+Statements.swift:440)
43  <MyApp>                    0x200f2dd74         [inlined] commit (Database.swift:1194)
44  <MyApp>                    0x200f2dd74         Database.inTransaction (Database.swift:987)
45  <MyApp>                    0x200f2e558         Database.inSavepoint (Database.swift:1066)
46  <MyApp>                    0x200f2e280         Database.isolated<T> (Database.swift:1016)
47  <MyApp>                    0x2010a6774         [inlined] ? (ValueObserver.swift:204)
48  <MyApp>                    0x2010a6774         ValueObserver.fetchFutureSync (ValueObserver.swift:203)
49  <MyApp>                    0x2010a619c         [inlined] fetchFuture
50  <MyApp>                    0x2010a619c         ValueObserver.databaseDidCommit (ValueObserver.swift:100)
51  <MyApp>                    0x200fb7f9c         [inlined] databaseDidCommit (TransactionObserver.swift:850)
52  <MyApp>                    0x200fb7f9c         DatabaseObservationBroker.databaseDidCommit (TransactionObserver.swift:447)
53  <MyApp>                    0x200fb6c98         [inlined] databaseDidCommitEmptyDeferredTransaction (TransactionObserver.swift:490)
54  <MyApp>                    0x200fb6c98         DatabaseObservationBroker.statementDidExecute (TransactionObserver.swift:346)
55  <MyApp>                    0x200f9c46c         [inlined] statementDidExecute (Database+Statements.swift:570)
56  <MyApp>                    0x200f9c46c         Statement.execute (Statement.swift:304)
57  <MyApp>                    0x200f2c218         Database.withAuthorizer<T> (<compiler-generated>:592)
58  <MyApp>                    0x200f246fc         [inlined] execute (Statement.swift:299)
59  <MyApp>                    0x200f246fc         Database.execute (Database+Statements.swift:465)
60  <MyApp>                    0x200f2dd74         [inlined] execute (Database+Statements.swift:440)
61  <MyApp>                    0x200f2dd74         [inlined] commit (Database.swift:1194)
62  <MyApp>                    0x200f2dd74         Database.inTransaction (Database.swift:987)
63  <MyApp>                    0x200f2e558         Database.inSavepoint (Database.swift:1066)
64  <MyApp>                    0x200f2e280         Database.isolated<T> (Database.swift:1016)
65  <MyApp>                    0x2010a6774         [inlined] ? (ValueObserver.swift:204)
66  <MyApp>                    0x2010a6774         ValueObserver.fetchFutureSync (ValueObserver.swift:203)
67  <MyApp>                    0x2010a619c         [inlined] fetchFuture
68  <MyApp>                    0x2010a619c         ValueObserver.databaseDidCommit (ValueObserver.swift:100)
69  <MyApp>                    0x200fb7f9c         [inlined] databaseDidCommit (TransactionObserver.swift:850)
70  <MyApp>                    0x200fb7f9c         DatabaseObservationBroker.databaseDidCommit (TransactionObserver.swift:447)
71  <MyApp>                    0x200fb6c98         [inlined] databaseDidCommitEmptyDeferredTransaction (TransactionObserver.swift:490)
72  <MyApp>                    0x200fb6c98         DatabaseObservationBroker.statementDidExecute (TransactionObserver.swift:346)
73  <MyApp>                    0x200f9c46c         [inlined] statementDidExecute (Database+Statements.swift:570)
74  <MyApp>                    0x200f9c46c         Statement.execute (Statement.swift:304)
75  <MyApp>                    0x200f2c218         Database.withAuthorizer<T> (<compiler-generated>:592)
76  <MyApp>                    0x200f246fc         [inlined] execute (Statement.swift:299)
77  <MyApp>                    0x200f246fc         Database.execute (Database+Statements.swift:465)
78  <MyApp>                    0x200f2dd74         [inlined] execute (Database+Statements.swift:440)
79  <MyApp>                    0x200f2dd74         [inlined] commit (Database.swift:1194)
80  <MyApp>                    0x200f2dd74         Database.inTransaction (Database.swift:987)
81  <MyApp>                    0x200f2e558         Database.inSavepoint (Database.swift:1066)
82  <MyApp>                    0x200f2e280         Database.isolated<T> (Database.swift:1016)
83  <MyApp>                    0x2010a6774         [inlined] ? (ValueObserver.swift:204)
84  <MyApp>                    0x2010a6774         ValueObserver.fetchFutureSync (ValueObserver.swift:203)
85  <MyApp>                    0x2010a619c         [inlined] fetchFuture
86  <MyApp>                    0x2010a619c         ValueObserver.databaseDidCommit (ValueObserver.swift:100)
87  <MyApp>                    0x200fb7f9c         [inlined] databaseDidCommit (TransactionObserver.swift:850)
88  <MyApp>                    0x200fb7f9c         DatabaseObservationBroker.databaseDidCommit (TransactionObserver.swift:447)
89  <MyApp>                    0x200fb6c98         [inlined] databaseDidCommitEmptyDeferredTransaction (TransactionObserver.swift:490)
90  <MyApp>                    0x200fb6c98         DatabaseObservationBroker.statementDidExecute (TransactionObserver.swift:346)
91  <MyApp>                    0x200f9c46c         [inlined] statementDidExecute (Database+Statements.swift:570)
92  <MyApp>                    0x200f9c46c         Statement.execute (Statement.swift:304)
93  <MyApp>                    0x200f2c218         Database.withAuthorizer<T> (<compiler-generated>:592)
94  <MyApp>                    0x200f246fc         [inlined] execute (Statement.swift:299)
95  <MyApp>                    0x200f246fc         Database.execute (Database+Statements.swift:465)
96  <MyApp>                    0x200f2dd74         [inlined] execute (Database+Statements.swift:440)
97  <MyApp>                    0x200f2dd74         [inlined] commit (Database.swift:1194)
98  <MyApp>                    0x200f2dd74         Database.inTransaction (Database.swift:987)
99  <MyApp>                    0x200f2e558         Database.inSavepoint (Database.swift:1066)
100 <MyApp>                    0x200f2e280         Database.isolated<T> (Database.swift:1016)
101 <MyApp>                    0x2010a6774         [inlined] ? (ValueObserver.swift:204)
102 <MyApp>                    0x2010a6774         ValueObserver.fetchFutureSync (ValueObserver.swift:203)
103 <MyApp>                    0x2010a619c         [inlined] fetchFuture
104 <MyApp>                    0x2010a619c         ValueObserver.databaseDidCommit (ValueObserver.swift:100)
105 <MyApp>                    0x200fb7f9c         [inlined] databaseDidCommit (TransactionObserver.swift:850)
106 <MyApp>                    0x200fb7f9c         DatabaseObservationBroker.databaseDidCommit (TransactionObserver.swift:447)
107 <MyApp>                    0x200fb6c98         [inlined] databaseDidCommitEmptyDeferredTransaction (TransactionObserver.swift:490)
108 <MyApp>                    0x200fb6c98         DatabaseObservationBroker.statementDidExecute (TransactionObserver.swift:346)
109 <MyApp>                    0x200f9c46c         [inlined] statementDidExecute (Database+Statements.swift:570)
110 <MyApp>                    0x200f9c46c         Statement.execute (Statement.swift:304)
111 <MyApp>                    0x200f2c218         Database.withAuthorizer<T> (<compiler-generated>:592)
112 <MyApp>                    0x200f246fc         [inlined] execute (Statement.swift:299)
113 <MyApp>                    0x200f246fc         Database.execute (Database+Statements.swift:465)
114 <MyApp>                    0x200f2dd74         [inlined] execute (Database+Statements.swift:440)
115 <MyApp>                    0x200f2dd74         [inlined] commit (Database.swift:1194)
116 <MyApp>                    0x200f2dd74         Database.inTransaction (Database.swift:987)
117 <MyApp>                    0x200f2e558         Database.inSavepoint (Database.swift:1066)
118 <MyApp>                    0x200f2e280         Database.isolated<T> (Database.swift:1016)
119 <MyApp>                    0x2010a6774         [inlined] ? (ValueObserver.swift:204)
120 <MyApp>                    0x2010a6774         ValueObserver.fetchFutureSync (ValueObserver.swift:203)
121 <MyApp>                    0x2010a619c         [inlined] fetchFuture
122 <MyApp>                    0x2010a619c         ValueObserver.databaseDidCommit (ValueObserver.swift:100)
123 <MyApp>                    0x200fb7f9c         [inlined] databaseDidCommit (TransactionObserver.swift:850)
124 <MyApp>                    0x200fb7f9c         DatabaseObservationBroker.databaseDidCommit (TransactionObserver.swift:447)
125 <MyApp>                    0x200fb6c98         [inlined] databaseDidCommitEmptyDeferredTransaction (TransactionObserver.swift:490)
126 <MyApp>                    0x200fb6c98         DatabaseObservationBroker.statementDidExecute (TransactionObserver.swift:346)
127 <MyApp>                    0x200f9c46c         [inlined] statementDidExecute (Database+Statements.swift:570)
128 <MyApp>                    0x200f9c46c         Statement.execute (Statement.swift:304)
129 <MyApp>                    0x200f2c218         Database.withAuthorizer<T> (<compiler-generated>:592)
130 <MyApp>                    0x200f246fc         [inlined] execute (Statement.swift:299)
131 <MyApp>                    0x200f246fc         Database.execute (Database+Statements.swift:465)
132 <MyApp>                    0x200f2dd74         [inlined] execute (Database+Statements.swift:440)
133 <MyApp>                    0x200f2dd74         [inlined] commit (Database.swift:1194)
134 <MyApp>                    0x200f2dd74         Database.inTransaction (Database.swift:987)
135 <MyApp>                    0x200f2e558         Database.inSavepoint (Database.swift:1066)
136 <MyApp>                    0x200f2e280         Database.isolated<T> (Database.swift:1016)
137 <MyApp>                    0x2010a6774         [inlined] ? (ValueObserver.swift:204)
138 <MyApp>                    0x2010a6774         ValueObserver.fetchFutureSync (ValueObserver.swift:203)
139 <MyApp>                    0x2010a619c         [inlined] fetchFuture
140 <MyApp>                    0x2010a619c         ValueObserver.databaseDidCommit (ValueObserver.swift:100)
141 <MyApp>                    0x200fb7f9c         [inlined] databaseDidCommit (TransactionObserver.swift:850)
142 <MyApp>                    0x200fb7f9c         DatabaseObservationBroker.databaseDidCommit (TransactionObserver.swift:447)
143 <MyApp>                    0x200fb6c98         [inlined] databaseDidCommitEmptyDeferredTransaction (TransactionObserver.swift:490)
144 <MyApp>                    0x200fb6c98         DatabaseObservationBroker.statementDidExecute (TransactionObserver.swift:346)
145 <MyApp>                    0x200f9c46c         [inlined] statementDidExecute (Database+Statements.swift:570)
146 <MyApp>                    0x200f9c46c         Statement.execute (Statement.swift:304)
147 <MyApp>                    0x200f2c218         Database.withAuthorizer<T> (<compiler-generated>:592)
148 <MyApp>                    0x200f246fc         [inlined] execute (Statement.swift:299)
149 <MyApp>                    0x200f246fc         Database.execute (Database+Statements.swift:465)
150 <MyApp>                    0x200f2dd74         [inlined] execute (Database+Statements.swift:440)
151 <MyApp>                    0x200f2dd74         [inlined] commit (Database.swift:1194)
152 <MyApp>                    0x200f2dd74         Database.inTransaction (Database.swift:987)
153 <MyApp>                    0x200f2e558         Database.inSavepoint (Database.swift:1066)
154 <MyApp>                    0x200f2e280         Database.isolated<T> (Database.swift:1016)
155 <MyApp>                    0x2010a6774         [inlined] ? (ValueObserver.swift:204)
156 <MyApp>                    0x2010a6774         ValueObserver.fetchFutureSync (ValueObserver.swift:203)
157 <MyApp>                    0x2010a619c         [inlined] fetchFuture
158 <MyApp>                    0x2010a619c         ValueObserver.databaseDidCommit (ValueObserver.swift:100)
159 <MyApp>                    0x200fb7f9c         [inlined] databaseDidCommit (TransactionObserver.swift:850)
160 <MyApp>                    0x200fb7f9c         DatabaseObservationBroker.databaseDidCommit (TransactionObserver.swift:447)
161 <MyApp>                    0x200fb6c98         [inlined] databaseDidCommitEmptyDeferredTransaction (TransactionObserver.swift:490)
162 <MyApp>                    0x200fb6c98         DatabaseObservationBroker.statementDidExecute (TransactionObserver.swift:346)

I should also add, that we use async/await and value observation throughout the app. In the beginning we had issues with value observation, as we incorrectly retained our view models due to a retain cycle and started too many observations at once. That also made this issue pop up locally. We since made sure to fix those issues and didn't come across this locally ever again. This is the only hint I have about the origin of this bug...

groue commented 2 years ago

Hello @xmanu,

Stack overflow in __cxa_throw

It looks like your app is entering an infinite loop.

I unfortunately cannot provide a simplified demo project as we cannot even reproduce the bug locally.

You will have to if you want help.

xmanu commented 2 years ago

Hi @groue,

It turned out that our app did have a retain issue that prevented view models from deallocating. This obviously shouldn't happen but it showed that GRDB has issues with high amounts of value observations.

I managed to reproduce the issue in a sample project. This sample project will create 100 concurrent value observations and the app will crash when new data is inserted into the observed table. The actual value after which the app crashes is lower (more like 74) and a real project would probably want to prevent observing a database so often, but still the GRDB framework should IMHO not produce stack overflows when an arbitrary observation limit is reached. The fascinating thing is that on initial app start (before inserting into the database) the 100 observations go through without a hitch, only after inserting the app crashes.

The sample code can be found here: https://github.com/xmanu/GRDBBug

Best, Manuel

groue commented 2 years ago

Thank you very much @xmanu.

So pathological application behavior can push GRBD out of its bounds - or rather the memory, since your app was facing a stack overflow.

I do not know yet if this deserves a fix in the library. I forked your sample code in https://github.com/groue/GRDB-Issue1209 so that it is not lost.

I'll keep this issue open a few days, until I decide if something has to be done or not.

xmanu commented 2 years ago

The issue in the sample project I keep getting is Thread 67: EXC_BAD_ACCESS (code=2, address=0x171007ff0). It doesn't say stack overflow in the debugger. It seems to be some kind of infinite loop though, as the stack trace has more than 1200 entries. Xcode never shows my app using more than 70 mb of memory...

It would be nice to have at least some kind of documentation on the issue then. A complex app with a deep navigation history could easily hit this limit when relying heavily on value observation to stay reactive. This can be probably prevented by sharing observations, but this is also not always easy to implement.

groue commented 2 years ago

It doesn't say stack overflow in the debugger.

Maybe that's just a difference between a application that runs in the debugger, and a released application.

A complex app with a deep navigation history could easily hit this limit when relying heavily on value observation to stay reactive.

I understand this point of view. I also think that an app that needs hundreds of observations to properly run is seriously conflating convenience with reasonable expectations. You're dealing with I/O here, not some abstract ideal oracle with infinite speed and resources. Tightening up your database accesses may be a good idea.

I'll add that a sentence like "I unfortunately cannot provide a simplified demo project" sometimes means that the application is out of control of the developers. Don't take this as an offense: we all do that. But this should not be considered as a good state of affairs, and certainly not the direct sign of a problem in a third-party library.

This explains why the priority of your report won't be high until I could have a look at your sample project.

This can be probably prevented by sharing observations, but this is also not always easy to implement.

If you have some specific feedback on possible improvements on the SharedValueObservation api, I'm quite willing to listen! πŸ‘

xmanu commented 2 years ago

I'll add that a sentence like "I unfortunately cannot provide a simplified demo project" sometimes means that the application is out of control of the developers. Don't take this as an offense: we all do that. But this should not be considered as a good state of affairs, and certainly not the direct sign of a problem in a third-party library.

This reads a little weird as I already managed to reduce this issue and provide a concise sample project and deflects from the issue at hand.

I originally created this issue to maybe invoke some feedback in tracking the crashes in our app down further. As of the time of creation of the issue I was still in the process of tracing the issue, which turned out to be an internal SwiftUI bug creating retain cycles (https://stackoverflow.com/q/71447839).

I am not advocating for using value observation excessively, and we didn't in our app, but 100 observations on a database with very few entries should not result in app crashes. The memory usage of the demo app is moderate (~70mb), and as I already commented above, the observation only fails when inserting into the table and the initial observation works fine.

It would be really nice if you could look into this issue as in my eyes it looks like the library has an infinite loop (somewhere, I am not deep enough into the internals. But the stack trace repeats every 20 or so entries, all in GRDB source code).

#41 0x0000000102f61f18 in TransactionObservation.databaseDidCommit(_:) at GRDB.swift/GRDB/Core/TransactionObserver.swift:846
#42 0x0000000102f61e2c in DatabaseObservationBroker.databaseDidCommit() at GRDB.swift/GRDB/Core/TransactionObserver.swift:443
#43 0x0000000102f62038 in DatabaseObservationBroker.databaseDidCommitEmptyDeferredTransaction() at GRDB.swift/GRDB/Core/TransactionObserver.swift:486
#44 0x0000000102f60c80 in DatabaseObservationBroker.statementDidExecute(_:) at GRDB.swift/GRDB/Core/TransactionObserver.swift:342
#45 0x0000000102e80b34 in Database.statementDidExecute(_:) at GRDB.swift/GRDB/Core/Database+Statements.swift:570
#46 0x0000000102f2d9a8 in Statement.execute(arguments:) at GRDB.swift/GRDB/Core/Statement.swift:307
#47 0x0000000102e7f6ac in Database.execute(literal:) at GRDB.swift/GRDB/Core/Database+Statements.swift:465
#48 0x0000000102e7f598 in Database.execute(sql:arguments:) at GRDB.swift/GRDB/Core/Database+Statements.swift:440
#49 0x0000000102e8d218 in Database.commit() at GRDB.swift/GRDB/Core/Database.swift:1186
#50 0x0000000102e8caec in Database.inTransaction(_:_:) at GRDB.swift/GRDB/Core/Database.swift:977
#51 0x0000000102e8df0c in Database.inSavepoint(_:) at GRDB.swift/GRDB/Core/Database.swift:1058
#52 0x0000000102e8daf4 in Database.isolated<Ο„_0_0>(readOnly:_:) at GRDB.swift/GRDB/Core/Database.swift:1008
#53 0x00000001030e0754 in closure #3 in ValueConcurrentObserver.databaseDidCommit(_:) at GRDB.swift/GRDB/ValueObservation/ValueConcurrentObserver.swift:652
#54 0x00000001030e2804 in partial apply for closure #3 in ValueConcurrentObserver.databaseDidCommit(_:) ()
#55 0x0000000102e89d04 in Database.recordingSelection<Ο„_0_0>(_:_:) at GRDB.swift/GRDB/Core/Database.swift:601
#56 0x00000001030dfc0c in ValueConcurrentObserver.databaseDidCommit(_:) at GRDB.swift/GRDB/ValueObservation/ValueConcurrentObserver.swift:651
#57 0x00000001030e1844 in protocol witness for TransactionObserver.databaseDidCommit(_:) in conformance ValueConcurrentObserver<Ο„_0_0> ()
#58 0x0000000102f61f18 in TransactionObservation.databaseDidCommit(_:) at GRDB.swift/GRDB/Core/TransactionObserver.swift:846
#59 0x0000000102f61e2c in DatabaseObservationBroker.databaseDidCommit() at GRDB.swift/GRDB/Core/TransactionObserver.swift:443
#60 0x0000000102f62038 in DatabaseObservationBroker.databaseDidCommitEmptyDeferredTransaction() at GRDB.swift/GRDB/Core/TransactionObserver.swift:486
#61 0x0000000102f60c80 in DatabaseObservationBroker.statementDidExecute(_:) at GRDB.swift/GRDB/Core/TransactionObserver.swift:342
#62 0x0000000102e80b34 in Database.statementDidExecute(_:) at GRDB.swift/GRDB/Core/Database+Statements.swift:570
#63 0x0000000102f2d9a8 in Statement.execute(arguments:) at GRDB.swift/GRDB/Core/Statement.swift:307
#64 0x0000000102e7f6ac in Database.execute(literal:) at GRDB.swift/GRDB/Core/Database+Statements.swift:465
#65 0x0000000102e7f598 in Database.execute(sql:arguments:) at GRDB.swift/GRDB/Core/Database+Statements.swift:440
#66 0x0000000102e8d218 in Database.commit() at GRDB.swift/GRDB/Core/Database.swift:1186
#67 0x0000000102e8caec in Database.inTransaction(_:_:) at GRDB.swift/GRDB/Core/Database.swift:977
#68 0x0000000102e8df0c in Database.inSavepoint(_:) at GRDB.swift/GRDB/Core/Database.swift:1058
#69 0x0000000102e8daf4 in Database.isolated<Ο„_0_0>(readOnly:_:) at GRDB.swift/GRDB/Core/Database.swift:1008
#70 0x00000001030e0754 in closure #3 in ValueConcurrentObserver.databaseDidCommit(_:) at GRDB.swift/GRDB/ValueObservation/ValueConcurrentObserver.swift:652
#71 0x00000001030e2804 in partial apply for closure #3 in ValueConcurrentObserver.databaseDidCommit(_:) ()
#72 0x0000000102e89d04 in Database.recordingSelection<Ο„_0_0>(_:_:) at GRDB.swift/GRDB/Core/Database.swift:601
#73 0x00000001030dfc0c in ValueConcurrentObserver.databaseDidCommit(_:) at GRDB.swift/GRDB/ValueObservation/ValueConcurrentObserver.swift:651

This excerpt of the stack trace repeats for about 1200 lines...

If you have some specific feedback on possible improvements on the SharedValueObservation api, I'm quite willing to listen!

I found that sharing an observation for a request that is parameterised is not as easy. For example implementing something like observeAllBooks(by authorId: Int64) -> AnyPublisher<[Book], Error> etc... To reuse this observation one would have to keep track of possible authorIds and keep an observation for each one (or limit it to the last one called, which might be undesired). I do not have a nice way of expressing this (and I don't know if this should even be done), but that is why we didn't so far use shared observations in our app.

groue commented 2 years ago

This reads a little weird as I already managed to reduce this issue and provide a concise sample project and deflects from the issue at hand.

You're right, and thank you for that. Yes, my sentence was not well balanced. It was not targeting you specifically, just expressing a general impression that I get sometimes from users who complain here about their own inability to ride their horse. This impression leaked into our conversation, I'm sorry for that.

I am not advocating for using value observation excessively, and we didn't in our app, but 100 observations on a database with very few entries should not result in app crashes

Yes, I read that. As I said above, I'll look at your reproducing application. I'll then decide on the action that should be done.

I found that sharing an observation for a request that is parameterised is not as easy. For example implementing something like observeAllBooks(by authorId: Int64) -> AnyPublisher<[Book], Error> etc... To reuse this observation one would have to keep track of possible authorIds and keep an observation for each one (or limit it to the last one called, which might be undesired). I do not have a nice way of expressing this (and I don't know if this should even be done), but that is why we didn't so far use shared observations in our app.

It's good to make the difference between your app, and a library that lets you address the needs of your app.

GRBD is a library, not your app.

When you know how to track books by author id, you'll know how to track authors by id, or books by country, or whatever. Trust your own skills: you should overcome this difficulty without much pain! You're getting more familiar with GRDB, and that's good!

groue commented 2 years ago

Hello again, @xmanu. I could reproduce the crash with your demo app. And you have revealed a GRDB bug indeed! I'm starting to understand it, and a fix will ship soon.

Thanks for having pushed the library off the tested limits πŸ‘

groue commented 2 years ago

πŸš€ The fix (#1213) has shipped in v5.24.0

xmanu commented 2 years ago

Thanks for looking into this and implementing a fix quickly! πŸ‘ Regarding the other issue starting to appear after 600 concurrent observations, I would say that this limit should most probably not be reached in a well architected app πŸ˜‰.

groue commented 2 years ago

Exactly πŸ˜„ It's also good when there's still improvements to bring AND there's no need to rush (i.e. nobody is blocked)πŸ˜‰