search

laurent22 / joplin

Joplin - the privacy-focused note taking app with sync capabilities for Windows, macOS, Linux, Android and iOS.
https://joplinapp.org
Other
45.02k stars 4.9k forks source link

Crash in iOS 16.02 when initializing sync #6923

Open drewmoseley opened 1 year ago

drewmoseley commented 1 year ago

I recently deleted the app from my phone to start fresh with a new sync target. I have my data syncing from the desktop app to the new S3-based sync target (https://app.idrivee2.com/) and it syncs fine between 2 desktops and an iPad. I probably should just have changed the sync target without deleting (that worked fine on my iPad) but alas I did not. Now whenever I setup the sync config and start to sync, the app crashes. If I relaunch it, it crashes again.

Environment

Joplin version: 12.8.1 Platform: iPhone iOS 16.0.2 OS specifics:

Steps to reproduce

  1. Remove the app
  2. Install the app
  3. Configure sync
  4. Start a sync

Describe what you expected to happen

Proper syncing.

Logfile

(I'm working on getting this but it seems they are not available in the documented location)

drewmoseley commented 1 year ago

Crash log from xcode: ` Incident Identifier: A74E9A54-7651-411E-84FB-5329382126D3 CrashReporter Key: c9ab59b7a46e9d6eb42ecac5fee99980e6e9bb5a Hardware Model: iPhone13,2 Process: Joplin [371] Path: /private/var/containers/Bundle/Application/F619D3DD-F490-43AE-9602-33D45852ADAF/Joplin.app/Joplin Identifier: net.cozic.joplin Version: 12.8.1 (82) AppStoreTools: 13F15 AppVariant: 1:iPhone13,2:15 Code Type: ARM-64 (Native) Role: Foreground Parent Process: launchd [1] Coalition: net.cozic.joplin [521]

Date/Time: 2022-10-06 21:28:53.9186 -0400 Launch Time: 2022-10-06 21:28:51.3132 -0400 OS Version: iPhone OS 16.0.2 (20A380) Release Type: User Baseband Version: 3.07.00 Report Version: 104

Exception Type: EXC_CRASH (SIGABRT) Exception Codes: 0x0000000000000000, 0x0000000000000000 Triggered by Thread: 3

Application Specific Information: abort() called

Last Exception Backtrace: 0 CoreFoundation 0x193a72248 exceptionPreprocess + 164 1 libobjc.A.dylib 0x18ce37a68 objc_exception_throw + 60 2 CoreFoundation 0x193be63f0 +[NSObject(NSObject) copyDescription] + 0 3 CoreFoundation 0x193a88360 forwarding_ + 1592 4 CoreFoundation 0x193af0660 _CF_forwarding_prep_0 + 96 5 Joplin 0x100930974 0x100854000 + 903540 6 CoreFoundation 0x193adca14 _invoking + 148 7 CoreFoundation 0x193a88f2c -[NSInvocation invoke] + 428 8 CoreFoundation 0x193a88944 -[NSInvocation invokeWithTarget:] + 64 9 Joplin 0x1008cb184 0x100854000 + 487812 10 Joplin 0x1008cd778 0x100854000 + 497528 11 Joplin 0x1008cd3a8 0x100854000 + 496552 12 libdispatch.dylib 0x19aeec4b4 _dispatch_call_block_and_release + 32 13 libdispatch.dylib 0x19aeedfdc _dispatch_client_callout + 20 14 libdispatch.dylib 0x19aef5694 _dispatch_lane_serial_drain + 672 15 libdispatch.dylib 0x19aef61e0 _dispatch_lane_invoke + 384 16 libdispatch.dylib 0x19af00e10 _dispatch_workloop_worker_thread + 652 17 libsystem_pthread.dylib 0x1e0390df8 _pthread_wqthread + 288 18 libsystem_pthread.dylib 0x1e0390b98 start_wqthread + 8

Thread 0 name: Dispatch queue: com.apple.main-thread Thread 0: 0 libsystem_kernel.dylib 0x1d0153b48 mach_msg2_trap + 8 1 libsystem_kernel.dylib 0x1d0166008 mach_msg2_internal + 80 2 libsystem_kernel.dylib 0x1d0166248 mach_msg_overwrite + 388 3 libsystem_kernel.dylib 0x1d015408c mach_msg + 24 4 CoreFoundation 0x193ae2e00 CFRunLoopServiceMachPort + 160 5 CoreFoundation 0x193ae4044 CFRunLoopRun + 1232 6 CoreFoundation 0x193ae91e4 CFRunLoopRunSpecific + 612 7 GraphicsServices 0x1cc909368 GSEventRunModal + 164 8 UIKitCore 0x195f98d88 -[UIApplication _run] + 888 9 UIKitCore 0x195f989ec UIApplicationMain + 340 10 Joplin 0x10085945c 0x100854000 + 21596 11 dyld 0x1b1e0d948 start + 2504

Thread 1: 0 libsystem_pthread.dylib 0x1e0390b90 start_wqthread + 0

Thread 2: 0 libsystem_pthread.dylib 0x1e0390b90 start_wqthread + 0

Thread 3 name: Dispatch queue: com.facebook.react.FileReaderModuleQueue Thread 3 Crashed: 0 libsystem_kernel.dylib 0x1d015a200 __pthread_kill + 8 1 libsystem_pthread.dylib 0x1e03971ac pthread_kill + 268 2 libsystem_c.dylib 0x19af51ca0 abort + 180 3 libc++abi.dylib 0x1e02d9b8c abort_message + 132 4 libc++abi.dylib 0x1e02c9a80 demangling_terminate_handler() + 336 5 libobjc.A.dylib 0x18ce3deec _objc_terminate() + 144 6 libc++abi.dylib 0x1e02d8f28 std::__terminate(void (*)()) + 20 7 libc++abi.dylib 0x1e02d8ec4 std::terminate() + 56 8 libdispatch.dylib 0x19aeedff0 _dispatch_client_callout + 40 9 libdispatch.dylib 0x19aef5694 _dispatch_lane_serial_drain + 672 10 libdispatch.dylib 0x19aef61e0 _dispatch_lane_invoke + 384 11 libdispatch.dylib 0x19af00e10 _dispatch_workloop_worker_thread + 652 12 libsystem_pthread.dylib 0x1e0390df8 _pthread_wqthread + 288 13 libsystem_pthread.dylib 0x1e0390b98 start_wqthread + 8

Thread 4 name: com.apple.uikit.eventfetch-thread Thread 4: 0 libsystem_kernel.dylib 0x1d0153b48 mach_msg2_trap + 8 1 libsystem_kernel.dylib 0x1d0166008 mach_msg2_internal + 80 2 libsystem_kernel.dylib 0x1d0166248 mach_msg_overwrite + 388 3 libsystem_kernel.dylib 0x1d015408c mach_msg + 24 4 CoreFoundation 0x193ae2e00 CFRunLoopServiceMachPort + 160 5 CoreFoundation 0x193ae4044 CFRunLoopRun + 1232 6 CoreFoundation 0x193ae91e4 CFRunLoopRunSpecific + 612 7 Foundation 0x18def1818 -[NSRunLoop(NSRunLoop) runMode:beforeDate:] + 212 8 Foundation 0x18def1700 -[NSRunLoop(NSRunLoop) runUntilDate:] + 64 9 UIKitCore 0x1960cd88c -[UIEventFetcher threadMain] + 436 10 Foundation 0x18df0ace8 NSThreadstart__ + 716 11 libsystem_pthread.dylib 0x1e03916cc _pthread_start + 148 12 libsystem_pthread.dylib 0x1e0390ba4 thread_start + 8

Thread 5: 0 libsystem_pthread.dylib 0x1e0390b90 start_wqthread + 0

Thread 6 name: Dispatch queue: com.apple.root.background-qos Thread 6: 0 CoreFoundation 0x193b01a70 _CFXNotificationRegistrarAddObserver + 140 1 CoreFoundation 0x193afbc9c CFXNotificationRegistrarAdd + 580 2 CoreFoundation 0x193afbc9c CFXNotificationRegistrarAdd + 580 3 CoreFoundation 0x193b12e5c _CFXNotificationRegisterObserver + 248 4 libdispatch.dylib 0x19aeec4b4 _dispatch_call_block_and_release + 32 5 libdispatch.dylib 0x19aeedfdc _dispatch_client_callout + 20 6 libdispatch.dylib 0x19aeffb8c _dispatch_root_queue_drain + 684 7 libdispatch.dylib 0x19af00284 _dispatch_worker_thread2 + 164 8 libsystem_pthread.dylib 0x1e0390dbc _pthread_wqthread + 228 9 libsystem_pthread.dylib 0x1e0390b98 start_wqthread + 8

Thread 7 name: Dispatch queue: com.apple.CFNetwork.CacheDB-write Thread 7: 0 libobjc.A.dylib 0x18ce27f98 bool objc::DenseMapBase<objc::DenseMap<DisguisedPtr, objc::DenseMap<void const, objc::ObjcAssociation, objc::DenseMapValueInfo, objc::DenseMapInfo<void const>, objc::detail::DenseMapPair<void const, objc::ObjcAssociation> >, objc::DenseMapValueInfo<objc::DenseMap<void const, objc::ObjcAssociation, objc::DenseMapValueInfo, objc::DenseMapInfo<void const>, objc::detail::DenseMapPair<void const, objc::ObjcAssociation> > >, objc::DenseMapInfo<DisguisedPtr >, objc::detail::DenseMapPair<DisguisedPtr, objc::DenseMap<void const, objc::ObjcAssociation, objc::DenseMapValueInfo, objc::DenseMapInfo<void const>, objc::detail::DenseMapPair<void const, objc::ObjcAssociation> > > >, DisguisedPtr, objc::DenseMap<void const, objc::ObjcAssociation, objc::DenseMapValueInfo, objc::DenseMapInfo<void const>, objc::detail::DenseMapPair<void const, objc::ObjcAssociation> >, objc::DenseMapValueInfo<objc::DenseMap<void const, objc::ObjcAssociation, objc::DenseMapValueInfo, objc::DenseMapInfo<void const>, objc::detail::DenseMapPair<void const, objc::ObjcAssociation> > >, objc::DenseMapInfo<DisguisedPtr >, objc::detail::DenseMapPair<DisguisedPtr, objc::DenseMap<void const, objc::ObjcAssociation, objc::DenseMapValueInfo, objc::DenseMapInfo<void const>, objc::detail::DenseMapPair<void const, objc::ObjcAssociation> > > >::LookupBucketFor<DisguisedPtr >(DisguisedPtr const&, objc::detail::DenseMapPair<DisguisedPtr, objc::DenseMap<void const, objc::ObjcAssociation, objc::DenseMapValueInfo, objc::DenseMapInfo<void const>, objc::detail::DenseMapPair<void const, objc::ObjcAssociation> > > const&) const + 132 1 libobjc.A.dylib 0x18ce5120c _object_remove_associations + 116 2 libobjc.A.dylib 0x18ce2622c objc_destructInstance + 96 3 libdispatch.dylib 0x19aeece68 _dispatch_dispose + 224 4 libxpc.dylib 0x1e03f0994 _xpc_serializer_dispose + 80 5 libxpc.dylib 0x1e03fd9d0 -[OS_xpc_object dealloc] + 28 6 libxpc.dylib 0x1e03e38cc xpc_connection_send_message_with_reply_sync + 528 7 RunningBoardServices 0x1a0e20bf8 -[RBSXPCMessage sendToConnection:error:] + 328 8 RunningBoardServices 0x1a0e20938 -[RBSXPCMessage invokeOnConnection:withReturnCollectionClass:entryClass:error:] + 92 9 RunningBoardServices 0x1a0e29fe4 -[RBSXPCMessage invokeOnConnection:withReturnClass:error:] + 32 10 RunningBoardServices 0x1a0e249fc -[RBSConnection acquireAssertion:error:] + 296 11 RunningBoardServices 0x1a0e246b0 -[RBSAssertion acquireWithError:] + 176 12 CFNetwork 0x194c104f4 0x194be3000 + 185588 13 CFNetwork 0x194cb0a88 0x194be3000 + 842376 14 libdispatch.dylib 0x19aefd114 _dispatch_block_async_invoke2 + 148 15 libdispatch.dylib 0x19aeedfdc _dispatch_client_callout + 20 16 libdispatch.dylib 0x19aef5694 _dispatch_lane_serial_drain + 672 17 libdispatch.dylib 0x19aef6214 _dispatch_lane_invoke + 436 18 libdispatch.dylib 0x19af00e10 _dispatch_workloop_worker_thread + 652 19 libsystem_pthread.dylib 0x1e0390df8 _pthread_wqthread + 288 20 libsystem_pthread.dylib 0x1e0390b98 start_wqthread + 8

Thread 8 name: com.facebook.react.JavaScript Thread 8: 0 libsystem_kernel.dylib 0x1d0153b48 mach_msg2_trap + 8 1 libsystem_kernel.dylib 0x1d0166008 mach_msg2_internal + 80 2 libsystem_kernel.dylib 0x1d0166248 mach_msg_overwrite + 388 3 libsystem_kernel.dylib 0x1d015408c mach_msg + 24 4 CoreFoundation 0x193ae2e00 CFRunLoopServiceMachPort + 160 5 CoreFoundation 0x193ae4044 CFRunLoopRun + 1232 6 CoreFoundation 0x193ae91e4 CFRunLoopRunSpecific + 612 7 Joplin 0x1008abd98 0x100854000 + 359832 8 Foundation 0x18df0ace8 NSThreadstart__ + 716 9 libsystem_pthread.dylib 0x1e03916cc _pthread_start + 148 10 libsystem_pthread.dylib 0x1e0390ba4 thread_start + 8

Thread 9 name: JavaScriptCore libpas scavenger Thread 9: 0 libsystem_kernel.dylib 0x1d015441c __psynch_cvwait + 8 1 libsystem_pthread.dylib 0x1e039806c _pthread_cond_wait + 1232 2 JavaScriptCore 0x1a6e5bba8 scavenger_thread_main + 1164 3 libsystem_pthread.dylib 0x1e03916cc _pthread_start + 148 4 libsystem_pthread.dylib 0x1e0390ba4 thread_start + 8

Thread 10 name: Heap Helper Thread Thread 10: 0 libsystem_kernel.dylib 0x1d015441c __psynch_cvwait + 8 1 libsystem_pthread.dylib 0x1e039806c _pthread_cond_wait + 1232 2 JavaScriptCore 0x1a6db25c4 WTF::ParkingLot::parkConditionallyImpl(void const, WTF::ScopedLambda<bool ()> const&, WTF::ScopedLambda<void ()> const&, WTF::TimeWithDynamicClockType const&) + 1808 3 JavaScriptCore 0x1a6d75d68 bool WTF::Condition::waitUntilUnchecked(WTF::Lock&, WTF::TimeWithDynamicClockType const&) + 304 4 JavaScriptCore 0x1a6d761e4 WTF::Detail::CallableWrapper<WTF::AutomaticThread::start(WTF::AbstractLocker const&)::$_0, void>::call() + 248 5 JavaScriptCore 0x1a6dd4f78 WTF::Thread::entryPoint(WTF::Thread::NewThreadContext) + 352 6 JavaScriptCore 0x1a6dd71fc WTF::wtfThreadEntryPoint(void*) + 16 7 libsystem_pthread.dylib 0x1e03916cc _pthread_start + 148 8 libsystem_pthread.dylib 0x1e0390ba4 thread_start + 8

Thread 11 name: Heap Helper Thread Thread 11: 0 libsystem_kernel.dylib 0x1d015441c __psynch_cvwait + 8 1 libsystem_pthread.dylib 0x1e039806c _pthread_cond_wait + 1232 2 JavaScriptCore 0x1a6db25c4 WTF::ParkingLot::parkConditionallyImpl(void const, WTF::ScopedLambda<bool ()> const&, WTF::ScopedLambda<void ()> const&, WTF::TimeWithDynamicClockType const&) + 1808 3 JavaScriptCore 0x1a6d75d68 bool WTF::Condition::waitUntilUnchecked(WTF::Lock&, WTF::TimeWithDynamicClockType const&) + 304 4 JavaScriptCore 0x1a6d761e4 WTF::Detail::CallableWrapper<WTF::AutomaticThread::start(WTF::AbstractLocker const&)::$_0, void>::call() + 248 5 JavaScriptCore 0x1a6dd4f78 WTF::Thread::entryPoint(WTF::Thread::NewThreadContext) + 352 6 JavaScriptCore 0x1a6dd71fc WTF::wtfThreadEntryPoint(void*) + 16 7 libsystem_pthread.dylib 0x1e03916cc _pthread_start + 148 8 libsystem_pthread.dylib 0x1e0390ba4 thread_start + 8

Thread 12 name: Heap Helper Thread Thread 12: 0 libsystem_kernel.dylib 0x1d015441c __psynch_cvwait + 8 1 libsystem_pthread.dylib 0x1e039806c _pthread_cond_wait + 1232 2 JavaScriptCore 0x1a6db25c4 WTF::ParkingLot::parkConditionallyImpl(void const, WTF::ScopedLambda<bool ()> const&, WTF::ScopedLambda<void ()> const&, WTF::TimeWithDynamicClockType const&) + 1808 3 JavaScriptCore 0x1a6d75d68 bool WTF::Condition::waitUntilUnchecked(WTF::Lock&, WTF::TimeWithDynamicClockType const&) + 304 4 JavaScriptCore 0x1a6d761e4 WTF::Detail::CallableWrapper<WTF::AutomaticThread::start(WTF::AbstractLocker const&)::$_0, void>::call() + 248 5 JavaScriptCore 0x1a6dd4f78 WTF::Thread::entryPoint(WTF::Thread::NewThreadContext) + 352 6 JavaScriptCore 0x1a6dd71fc WTF::wtfThreadEntryPoint(void*) + 16 7 libsystem_pthread.dylib 0x1e03916cc _pthread_start + 148 8 libsystem_pthread.dylib 0x1e0390ba4 thread_start + 8

Thread 13 name: Heap Helper Thread Thread 13: 0 libsystem_kernel.dylib 0x1d015441c __psynch_cvwait + 8 1 libsystem_pthread.dylib 0x1e039806c _pthread_cond_wait + 1232 2 JavaScriptCore 0x1a6db25c4 WTF::ParkingLot::parkConditionallyImpl(void const, WTF::ScopedLambda<bool ()> const&, WTF::ScopedLambda<void ()> const&, WTF::TimeWithDynamicClockType const&) + 1808 3 JavaScriptCore 0x1a6d75d68 bool WTF::Condition::waitUntilUnchecked(WTF::Lock&, WTF::TimeWithDynamicClockType const&) + 304 4 JavaScriptCore 0x1a6d761e4 WTF::Detail::CallableWrapper<WTF::AutomaticThread::start(WTF::AbstractLocker const&)::$_0, void>::call() + 248 5 JavaScriptCore 0x1a6dd4f78 WTF::Thread::entryPoint(WTF::Thread::NewThreadContext) + 352 6 JavaScriptCore 0x1a6dd71fc WTF::wtfThreadEntryPoint(void*) + 16 7 libsystem_pthread.dylib 0x1e03916cc _pthread_start + 148 8 libsystem_pthread.dylib 0x1e0390ba4 thread_start + 8

Thread 14 name: Heap Helper Thread Thread 14: 0 libsystem_kernel.dylib 0x1d015441c __psynch_cvwait + 8 1 libsystem_pthread.dylib 0x1e039806c _pthread_cond_wait + 1232 2 JavaScriptCore 0x1a6db25c4 WTF::ParkingLot::parkConditionallyImpl(void const, WTF::ScopedLambda<bool ()> const&, WTF::ScopedLambda<void ()> const&, WTF::TimeWithDynamicClockType const&) + 1808 3 JavaScriptCore 0x1a6d75d68 bool WTF::Condition::waitUntilUnchecked(WTF::Lock&, WTF::TimeWithDynamicClockType const&) + 304 4 JavaScriptCore 0x1a6d761e4 WTF::Detail::CallableWrapper<WTF::AutomaticThread::start(WTF::AbstractLocker const&)::$_0, void>::call() + 248 5 JavaScriptCore 0x1a6dd4f78 WTF::Thread::entryPoint(WTF::Thread::NewThreadContext) + 352 6 JavaScriptCore 0x1a6dd71fc WTF::wtfThreadEntryPoint(void*) + 16 7 libsystem_pthread.dylib 0x1e03916cc _pthread_start + 148 8 libsystem_pthread.dylib 0x1e0390ba4 thread_start + 8

Thread 15 name: com.apple.CoreMotion.MotionThread Thread 15: 0 libsystem_kernel.dylib 0x1d0153b48 mach_msg2_trap + 8 1 libsystem_kernel.dylib 0x1d0166008 mach_msg2_internal + 80 2 libsystem_kernel.dylib 0x1d0166248 mach_msg_overwrite + 388 3 libsystem_kernel.dylib 0x1d015408c mach_msg + 24 4 CoreFoundation 0x193ae2e00 CFRunLoopServiceMachPort + 160 5 CoreFoundation 0x193ae4044 CFRunLoopRun + 1232 6 CoreFoundation 0x193ae91e4 CFRunLoopRunSpecific + 612 7 CoreFoundation 0x193b2d014 CFRunLoopRun + 64 8 CoreMotion 0x19ee64b04 0x19ee51000 + 80644 9 libsystem_pthread.dylib 0x1e03916cc _pthread_start + 148 10 libsystem_pthread.dylib 0x1e0390ba4 thread_start + 8

Thread 16 name: JSC Heap Collector Thread Thread 16: 0 libsystem_kernel.dylib 0x1d015441c __psynch_cvwait + 8 1 libsystem_pthread.dylib 0x1e039806c _pthread_cond_wait + 1232 2 JavaScriptCore 0x1a6db25c4 WTF::ParkingLot::parkConditionallyImpl(void const, WTF::ScopedLambda<bool ()> const&, WTF::ScopedLambda<void ()> const&, WTF::TimeWithDynamicClockType const&) + 1808 3 JavaScriptCore 0x1a6d75d68 bool WTF::Condition::waitUntilUnchecked(WTF::Lock&, WTF::TimeWithDynamicClockType const&) + 304 4 JavaScriptCore 0x1a6d761e4 WTF::Detail::CallableWrapper<WTF::AutomaticThread::start(WTF::AbstractLocker const&)::$_0, void>::call() + 248 5 JavaScriptCore 0x1a6dd4f78 WTF::Thread::entryPoint(WTF::Thread::NewThreadContext) + 352 6 JavaScriptCore 0x1a6dd71fc WTF::wtfThreadEntryPoint(void*) + 16 7 libsystem_pthread.dylib 0x1e03916cc _pthread_start + 148 8 libsystem_pthread.dylib 0x1e0390ba4 thread_start + 8

Thread 17 name: com.apple.NSURLConnectionLoader Thread 17: 0 libsystem_kernel.dylib 0x1d0153b48 mach_msg2_trap + 8 1 libsystem_kernel.dylib 0x1d0166008 mach_msg2_internal + 80 2 libsystem_kernel.dylib 0x1d0166248 mach_msg_overwrite + 388 3 libsystem_kernel.dylib 0x1d015408c mach_msg + 24 4 CoreFoundation 0x193ae2e00 CFRunLoopServiceMachPort + 160 5 CoreFoundation 0x193ae4044 CFRunLoopRun + 1232 6 CoreFoundation 0x193ae91e4 CFRunLoopRunSpecific + 612 7 CFNetwork 0x194e37bac 0x194be3000 + 2444204 8 Foundation 0x18df0ace8 NSThreadstart__ + 716 9 libsystem_pthread.dylib 0x1e03916cc _pthread_start + 148 10 libsystem_pthread.dylib 0x1e0390ba4 thread_start + 8

Thread 3 crashed with ARM Thread State (64-bit): x0: 0x0000000000000000 x1: 0x0000000000000000 x2: 0x0000000000000000 x3: 0x0000000000000000 x4: 0x00000001e02dd0f5 x5: 0x000000016f74a430 x6: 0x000000000000006e x7: 0xffffffff00000500 x8: 0x9f06513e7fd6da54 x9: 0x9f06513f10a26a54 x10: 0x0000000000000200 x11: 0x000000000000000b x12: 0x000000000000000b x13: 0x0000000000000000 x14: 0x0000000000000000 x15: 0x0000000000000000 x16: 0x0000000000000148 x17: 0x000000016f74b000 x18: 0x0000000000000000 x19: 0x0000000000000006 x20: 0x0000000000002403 x21: 0x000000016f74b0e0 x22: 0x0000000000000000 x23: 0x000000016f74b0e0 x24: 0x0000000000000000 x25: 0x000000028135e1e8 x26: 0x0000000000000114 x27: 0x0000000000000000 x28: 0x000000028285e840 fp: 0x000000016f74a3a0 lr: 0x00000001e03971ac sp: 0x000000016f74a380 pc: 0x00000001d015a200 cpsr: 0x40001000 far: 0x00000001e727f170 esr: 0x56000080 Address size fault

Binary Images: 0x1d0153000 - 0x1d0189ffb libsystem_kernel.dylib arm64e /usr/lib/system/libsystem_kernel.dylib 0x193a68000 - 0x193e4cfff CoreFoundation arm64e <42c5c91704473995b50fde4d132c2435> /System/Library/Frameworks/CoreFoundation.framework/CoreFoundation 0x1cc908000 - 0x1cc910fff GraphicsServices arm64e <03732ba5113235b4b09d8dd49807d246> /System/Library/PrivateFrameworks/GraphicsServices.framework/GraphicsServices 0x195bf6000 - 0x1973bcfff UIKitCore arm64e <7b942fa4cb7633759972f58c14492fb4> /System/Library/PrivateFrameworks/UIKitCore.framework/UIKitCore 0x100854000 - 0x100a87fff Joplin arm64 <14bf95e95db735f69719e873581792c1> /private/var/containers/Bundle/Application/F619D3DD-F490-43AE-9602-33D45852ADAF/Joplin.app/Joplin 0x1b1df8000 - 0x1b1e7ae4f dyld arm64e <341bbf646034357e8aa6e1e4b988e03c> /usr/lib/dyld 0x1e0390000 - 0x1e039bfff libsystem_pthread.dylib arm64e /usr/lib/system/libsystem_pthread.dylib 0x19af31000 - 0x19afb0fff libsystem_c.dylib arm64e <82c0f938e2773165a61470151c651798> /usr/lib/system/libsystem_c.dylib 0x1e02c8000 - 0x1e02dfffb libc++abi.dylib arm64e /usr/lib/libc++abi.dylib 0x18ce20000 - 0x18ce63c3f libobjc.A.dylib arm64e /usr/lib/libobjc.A.dylib 0x19aeea000 - 0x19af30fff libdispatch.dylib arm64e /usr/lib/system/libdispatch.dylib 0x18deb0000 - 0x18e7fafff Foundation arm64e /System/Library/Frameworks/Foundation.framework/Foundation 0x1e03d4000 - 0x1e0413fff libxpc.dylib arm64e <30ec91deda9035e184036402e6559bae> /usr/lib/system/libxpc.dylib 0x1a0e1c000 - 0x1a0e71fff RunningBoardServices arm64e <32ad4aa53d97332e8381ed24fa12301a> /System/Library/PrivateFrameworks/RunningBoardServices.framework/RunningBoardServices 0x194be3000 - 0x194fa8fff CFNetwork arm64e <35605de33723335a83d96f35f2989935> /System/Library/Frameworks/CFNetwork.framework/CFNetwork 0x1a6d68000 - 0x1a823ffff JavaScriptCore arm64e <9f4a8d8984473fa5a2e388d3c7ba1684> /System/Library/Frameworks/JavaScriptCore.framework/JavaScriptCore 0x19ee51000 - 0x19f230fff CoreMotion arm64e <74a270a3b1723245a4697225708d6386> /System/Library/Frameworks/CoreMotion.framework/CoreMotion

EOF

`

shivermetim commented 1 year ago

Very similar situation for me on iOS 16.1 beta 4, also beta 3. Also using S3. It syncs to a point and then can’t handle it. My existing sync when I upgraded to iOS 16.1 was not syncing (hence the reset) but the app was stable enough to use.

This is my second attempt to sync from scratch and it managed to string me along for a while — my documents and data for the iOS app = 4.39gb. Prior to this, it only managed to sync 500mb before the crash started.

I wonder if there could be a specific note or attachment that’s screwing it up? I’ll dig out my logs and let the smart people see if there’s any correlation.

drewmoseley commented 1 year ago

Strange; my iPad sync worked for a while but just today I noticed it was experiencing a similar failure. I'm not able to pull a crash log at the moment but will try to get one in the next few days.

shivermetim commented 1 year ago

Have you managed to make any headway? I've uninstalled and reinstalled Joplin several times with successive iOS updates, and it still crashes in 10 seconds. I'm extremely non-technical so I'm not sure what the next step is here.

drewmoseley commented 1 year ago

I switched back to dropbox syncing for now. When this bug is fixed, I'll try S3 syncing again.

TonyPepeBear commented 1 year ago

I got this issue too. S3 sync can work on my mac and android. But on iOS, the app will crash in about 3 seconds.

I am on 15.6.1.

shivermetim commented 1 year ago

If S3 is the issue I’m just going to wait until it’s resolved. Syncing with OneDrive was infinitely worse, and S3 has been completely painless until this. C’est La vie

leematos commented 1 year ago

Strange. I'm on iOS 16.1.1 and not getting these crashes with S3 sync on an iPhone 13 Pro. That said, I only have 60MB of data.

Looking at the crash log it seems like it's react-natives file reader that's upset:

Thread 3 name: Dispatch queue: com.facebook.react.FileReaderModuleQueue
Thread 3 Crashed:
0 libsystem_kernel.dylib 0x1d015a200 __pthread_kill + 8
1 libsystem_pthread.dylib 0x1e03971ac pthread_kill + 268
2 libsystem_c.dylib 0x19af51ca0 abort + 180
3 libc++abi.dylib 0x1e02d9b8c abort_message + 132
4 libc++abi.dylib 0x1e02c9a80 demangling_terminate_handler() + 336
5 libobjc.A.dylib 0x18ce3deec _objc_terminate() + 144
6 libc++abi.dylib 0x1e02d8f28 std::__terminate(void (*)()) + 20
7 libc++abi.dylib 0x1e02d8ec4 std::terminate() + 56
8 libdispatch.dylib 0x19aeedff0 _dispatch_client_callout + 40
9 libdispatch.dylib 0x19aef5694 _dispatch_lane_serial_drain + 672
10 libdispatch.dylib 0x19aef61e0 _dispatch_lane_invoke + 384
11 libdispatch.dylib 0x19af00e10 _dispatch_workloop_worker_thread + 652
12 libsystem_pthread.dylib 0x1e0390df8 _pthread_wqthread + 288
13 libsystem_pthread.dylib 0x1e0390b98 start_wqthread + 8

I have to suspect that it's "very large files" or "a collection of moderately large files that are not getting cleaned up properly and we are running out of memory or something like that.

Any chance you have a sense if either of those are the case? a mondo file (10GB+) or say many 2-3GB files?

drewmoseley commented 1 year ago

I don't think so. In my dropbox sync folder, the total size of all the data is 920MB.

leematos commented 1 year ago

Since we are initializing a sync, I suspect that the code is using a "get" call to download and the line that's relevant is this one: https://github.com/laurent22/joplin/blob/dev/packages/lib/file-api-driver-amazon-s3.js#L248

Which that function has a comment: https://github.com/laurent22/joplin/blob/1cfbefb76a4ef74cef3c7e013900953adf2e9719/packages/app-mobile/utils/shim-init-react.js#L154-L157

I don't see us actually doing the read there though. It's either:

https://github.com/laurent22/joplin/blob/dev/packages/lib/Synchronizer.ts#L622

or

https://github.com/laurent22/joplin/blob/dev/packages/lib/Synchronizer.ts#L900

Regardless it should be using BaseItem.unserialize: https://github.com/laurent22/joplin/blob/dev/packages/lib/models/BaseItem.ts#L477

Nothing there sticks out as problematic on the nose....

@laurent22 I'm trying to debug this S3 sync issue. Can you take a look at the steps above and tell me if I'm missing anything about how those blobs are getting deserialized deeper in joplin sync?

leematos commented 1 year ago

It's late here so I'm not 100% but it's possible that during an S3 sync for a larger media file, we get a response.type that isn't path and that triggers https://github.com/laurent22/joplin/blob/1cfbefb76a4ef74cef3c7e013900953adf2e9719/packages/app-mobile/utils/shim-init-react.js#L160

I won't have time for a bit to get debugging set up on that, but if someone else reading wants to take a look, I'd:

mariusv commented 1 year ago

I have the same sync issue even with an empty database. I have only one note which is a small table but still fails to sync and crashes

github-actions[bot] commented 1 year ago

Hey there, it looks like there has been no activity on this issue recently. Has the issue been fixed, or does it still require the community's attention? If you require support or are requesting an enhancement or feature then please create a topic on the Joplin forum. This issue may be closed if no further activity occurs. You may comment on the issue and I will leave it open. Thank you for your contributions.

mariusv commented 1 year ago

So far is still not fixed

wrein commented 1 year ago

So far is still not fixed

I can confirm there's no solution yet.

github-actions[bot] commented 1 year ago

Hey there, it looks like there has been no activity on this issue recently. Has the issue been fixed, or does it still require the community's attention? If you require support or are requesting an enhancement or feature then please create a topic on the Joplin forum. This issue may be closed if no further activity occurs. You may comment on the issue and I will leave it open. Thank you for your contributions.

mariusv commented 1 year ago

I'm afraid the issue is not solved. Also there was no new iOS release so please don't close this issue

cheyue commented 1 year ago

IOS 16.5.1 新安装配置S3 到密钥后立即闪退。望能更新。

all4fun commented 11 months ago

IOS 16.6.1 ,newly install, config s3 sync, still crash

leematos commented 11 months ago

@all4fun any more details?

  1. Can you share a log of the sync attempt?
  2. Are you using AWS or some other backend?

There is clearly some edge case we are missing here

kamilczerw commented 9 months ago

I'm getting the same error on iOS 17.2, on iPhone 12 Pro.

I'm trying to sync with self hosted MinIO.

Here is the crash report. 

{"app_name":"Joplin","timestamp":"2023-11-22 15:53:23.00 +0100","app_version":"12.12.3","slice_uuid":"b16a4957-a0ee-362a-9290-8f2f685c6c9e","adam_id":"1315599797","build_version":"97","platform":2,"bundleID":"net.cozic.joplin","share_with_app_devs":0,"is_first_party":0,"bug_type":"309","os_version":"iPhone OS 17.2 (21C5046c)","roots_installed":0,"name":"Joplin","incident_id":"3B433B26-3945-4042-BCF0-CAE062ECCCE2"}
{
  "uptime" : 200000,
  "procRole" : "Foreground",
  "version" : 2,
  "userID" : 501,
  "deployVersion" : 210,
  "modelCode" : "iPhone13,3",
  "coalitionID" : 1882,
  "osVersion" : {
    "isEmbedded" : true,
    "train" : "iPhone OS 17.2",
    "releaseType" : "Beta",
    "build" : "21C5046c"
  },
  "captureTime" : "2023-11-22 15:53:23.6940 +0100",
  "codeSigningMonitor" : 1,
  "incident" : "3B433B26-3945-4042-BCF0-CAE062ECCCE2",
  "pid" : 5544,
  "cpuType" : "ARM-64",
  "roots_installed" : 0,
  "bug_type" : "309",
  "procLaunch" : "2023-11-22 15:53:22.9211 +0100",
  "procStartAbsTime" : 4886388926236,
  "procExitAbsTime" : 4886407394628,
  "procName" : "Joplin",
  "procPath" : "\/private\/var\/containers\/Bundle\/Application\/63B03EA0-B884-4E51-93C5-2F81BA502DA8\/Joplin.app\/Joplin",
  "bundleInfo" : {"CFBundleShortVersionString":"12.12.3","CFBundleVersion":"97","CFBundleIdentifier":"net.cozic.joplin","DTAppStoreToolsBuild":"14E221"},
  "storeInfo" : {"storeCohortMetadata":"10|date=1700663400000&sf=143456&pgtp=Search&pgid=d1c062be-5d23-4ea6-8898-6c83a760cf7b&prpg=SearchLanding_SearchLanding&ctxt=Search&issrch=1&imptyp=card&kind=iosSoftware&lngid=2","itemID":"1315599797","deviceIdentifierForVendor":"EF9B35DB-39F3-44C4-A7F7-03622CE87653","softwareVersionExternalIdentifier":"859835060","thirdParty":true,"applicationVariant":"1:iPhone13,3:15"},
  "parentProc" : "launchd",
  "parentPid" : 1,
  "coalitionName" : "net.cozic.joplin",
  "crashReporterKey" : "73ebb41c173c9a2d4c35516eb451dd51ecc1f6da",
  "wasUnlockedSinceBoot" : 1,
  "isLocked" : 0,
  "codeSigningID" : "net.cozic.joplin",
  "codeSigningTeamID" : "A9BXAFS6CT",
  "codeSigningFlags" : 570450689,
  "codeSigningValidationCategory" : 4,
  "codeSigningTrustLevel" : 7,
  "instructionByteStream" : {"beforePC":"fyMD1f17v6n9AwCR893\/l78DAJH9e8Go\/w9f1sADX9YQKYDSARAA1A==","atPC":"AwEAVH8jA9X9e7+p\/QMAkejd\/5e\/AwCR\/XvBqP8PX9bAA1\/WfyMD1Q=="},
  "basebandVersion" : "4.20.05",
  "exception" : {"codes":"0x0000000000000000, 0x0000000000000000","rawCodes":[0,0],"type":"EXC_CRASH","signal":"SIGABRT"},
  "termination" : {"flags":0,"code":6,"namespace":"SIGNAL","indicator":"Abort trap: 6","byProc":"Joplin","byPid":5544},
  "asi" : {"libsystem_c.dylib":["abort() called"]},
  "exceptionReason" : {"arguments":["NSNull","length","0x1eff148b8"],"format_string":"-[%s %s]: unrecognized selector sent to instance %p","name":"NSInvalidArgumentException","type":"objc-exception","composed_message":"-[NSNull length]: unrecognized selector sent to instance 0x1eff148b8","class":"NSException"},
  "lastExceptionBacktrace" : [{"imageOffset":968800,"symbol":"__exceptionPreprocess","symbolLocation":164,"imageIndex":4},{"imageOffset":179328,"symbol":"objc_exception_throw","symbolLocation":60,"imageIndex":12},{"imageOffset":1560988,"symbol":"+[NSObject(NSObject) _copyDescription]","symbolLocation":0,"imageIndex":4},{"imageOffset":204760,"symbol":"___forwarding___","symbolLocation":1564,"imageIndex":4},{"imageOffset":1518352,"symbol":"_CF_forwarding_prep_0","symbolLocation":96,"imageIndex":4},{"imageOffset":908168,"imageIndex":2},{"imageOffset":9896,"symbol":"_dispatch_call_block_and_release","symbolLocation":32,"imageIndex":13},{"imageOffset":17152,"symbol":"_dispatch_client_callout","symbolLocation":20,"imageIndex":13},{"imageOffset":47252,"symbol":"_dispatch_lane_serial_drain","symbolLocation":748,"imageIndex":13},{"imageOffset":50116,"symbol":"_dispatch_lane_invoke","symbolLocation":380,"imageIndex":13},{"imageOffset":94212,"symbol":"_dispatch_root_queue_drain_deferred_wlh","symbolLocation":288,"imageIndex":13},{"imageOffset":92280,"symbol":"_dispatch_workloop_worker_thread","symbolLocation":404,"imageIndex":13},{"imageOffset":6500,"symbol":"_pthread_wqthread","symbolLocation":288,"imageIndex":10},{"imageOffset":6660,"symbol":"start_wqthread","symbolLocation":8,"imageIndex":10}],
  "faultingThread" : 3,
  "threads" : [{"id":2878435,"threadState":{"x":[{"value":268451845},{"value":21592279046},{"value":8589934592},{"value":24202140712960},{"value":0},{"value":24202140712960},{"value":2},{"value":4294967295},{"value":18446744073709550527},{"value":5635},{"value":0},{"value":1},{"value":5635},{"value":561573},{"value":1099511627776},{"value":0},{"value":18446744073709551569},{"value":7003640036,"symbolLocation":56,"symbol":"clock_gettime"},{"value":0},{"value":4294967295},{"value":2},{"value":24202140712960},{"value":0},{"value":24202140712960},{"value":6127708104},{"value":8589934592},{"value":21592279046},{"value":21592279046},{"value":4412409862}],"flavor":"ARM_THREAD_STATE64","lr":{"value":8067550992},"cpsr":{"value":4096},"fp":{"value":6127707952},"sp":{"value":6127707872},"esr":{"value":1442840704,"description":" Address size fault"},"pc":{"value":8067551608},"far":{"value":0}},"queue":"com.apple.main-thread","frames":[{"imageOffset":4472,"symbol":"mach_msg2_trap","symbolLocation":8,"imageIndex":3},{"imageOffset":3856,"symbol":"mach_msg2_internal","symbolLocation":80,"imageIndex":3},{"imageOffset":3624,"symbol":"mach_msg_overwrite","symbolLocation":436,"imageIndex":3},{"imageOffset":3176,"symbol":"mach_msg","symbolLocation":24,"imageIndex":3},{"imageOffset":220396,"symbol":"__CFRunLoopServiceMachPort","symbolLocation":160,"imageIndex":4},{"imageOffset":211940,"symbol":"__CFRunLoopRun","symbolLocation":1208,"imageIndex":4},{"imageOffset":210504,"symbol":"CFRunLoopRunSpecific","symbolLocation":608,"imageIndex":4},{"imageOffset":13560,"symbol":"GSEventRunModal","symbolLocation":164,"imageIndex":5},{"imageOffset":2281432,"symbol":"-[UIApplication _run]","symbolLocation":888,"imageIndex":6},{"imageOffset":2278932,"symbol":"UIApplicationMain","symbolLocation":340,"imageIndex":6},{"imageOffset":22972,"imageIndex":2},{"imageOffset":24012,"symbol":"start","symbolLocation":2240,"imageIndex":7}]},{"id":2878436,"frames":[{"imageOffset":6652,"symbol":"start_wqthread","symbolLocation":0,"imageIndex":10}],"threadState":{"x":[{"value":6128267264},{"value":7427},{"value":6127730688},{"value":0},{"value":409604},{"value":18446744073709551615},{"value":0},{"value":0},{"value":0},{"value":0},{"value":0},{"value":0},{"value":0},{"value":0},{"value":0},{"value":0},{"value":0},{"value":0},{"value":0},{"value":0},{"value":0},{"value":0},{"value":0},{"value":0},{"value":0},{"value":0},{"value":0},{"value":0},{"value":0}],"flavor":"ARM_THREAD_STATE64","lr":{"value":0},"cpsr":{"value":4096},"fp":{"value":0},"sp":{"value":6128267264},"esr":{"value":1442840704,"description":" Address size fault"},"pc":{"value":8646916604},"far":{"value":0}}},{"id":2878437,"frames":[{"imageOffset":6652,"symbol":"start_wqthread","symbolLocation":0,"imageIndex":10}],"threadState":{"x":[{"value":6128840704},{"value":4355},{"value":6128304128},{"value":0},{"value":409604},{"value":18446744073709551615},{"value":0},{"value":0},{"value":0},{"value":0},{"value":0},{"value":0},{"value":0},{"value":0},{"value":0},{"value":0},{"value":0},{"value":0},{"value":0},{"value":0},{"value":0},{"value":0},{"value":0},{"value":0},{"value":0},{"value":0},{"value":0},{"value":0},{"value":0}],"flavor":"ARM_THREAD_STATE64","lr":{"value":0},"cpsr":{"value":4096},"fp":{"value":0},"sp":{"value":6128840704},"esr":{"value":1442840704,"description":" Address size fault"},"pc":{"value":8646916604},"far":{"value":0}}},{"triggered":true,"id":2878438,"threadState":{"x":[{"value":0},{"value":0},{"value":0},{"value":0},{"value":8646079343},{"value":6129412112},{"value":110},{"value":0},{"value":9190933707150351225},{"value":9190933711121230713},{"value":512},{"value":6129410880},{"value":0},{"value":2095104},{"value":16},{"value":0},{"value":328},{"value":6129414144},{"value":0},{"value":6},{"value":9475},{"value":6129414368},{"value":276},{"value":6129414368},{"value":10786066152},{"value":0},{"value":0},{"value":10759839744},{"value":0}],"flavor":"ARM_THREAD_STATE64","lr":{"value":8646932096},"cpsr":{"value":1073745920},"fp":{"value":6129411968},"sp":{"value":6129411936},"esr":{"value":1442840704,"description":" Address size fault"},"pc":{"value":8067588028,"matchesCrashFrame":1},"far":{"value":0}},"queue":"com.facebook.react.BlobModuleQueue","frames":[{"imageOffset":40892,"symbol":"__pthread_kill","symbolLocation":8,"imageIndex":3},{"imageOffset":22144,"symbol":"pthread_kill","symbolLocation":268,"imageIndex":10},{"imageOffset":482192,"symbol":"abort","symbolLocation":180,"imageIndex":9},{"imageOffset":86008,"symbol":"abort_message","symbolLocation":132,"imageIndex":11},{"imageOffset":20368,"symbol":"demangling_terminate_handler()","symbolLocation":348,"imageIndex":11},{"imageOffset":191908,"symbol":"_objc_terminate()","symbolLocation":144,"imageIndex":12},{"imageOffset":82876,"symbol":"std::__terminate(void (*)())","symbolLocation":16,"imageIndex":11},{"imageOffset":82784,"symbol":"std::terminate()","symbolLocation":108,"imageIndex":11},{"imageOffset":17172,"symbol":"_dispatch_client_callout","symbolLocation":40,"imageIndex":13},{"imageOffset":47252,"symbol":"_dispatch_lane_serial_drain","symbolLocation":748,"imageIndex":13},{"imageOffset":50116,"symbol":"_dispatch_lane_invoke","symbolLocation":380,"imageIndex":13},{"imageOffset":94212,"symbol":"_dispatch_root_queue_drain_deferred_wlh","symbolLocation":288,"imageIndex":13},{"imageOffset":92280,"symbol":"_dispatch_workloop_worker_thread","symbolLocation":404,"imageIndex":13},{"imageOffset":6500,"symbol":"_pthread_wqthread","symbolLocation":288,"imageIndex":10},{"imageOffset":6660,"symbol":"start_wqthread","symbolLocation":8,"imageIndex":10}]},{"id":2878439,"name":"com.apple.uikit.eventfetch-thread","threadState":{"x":[{"value":268451845},{"value":21592279046},{"value":8589934592},{"value":69282117451776},{"value":3640971268},{"value":69282117451776},{"value":2},{"value":4294967295},{"value":18446744073709550527},{"value":16131},{"value":0},{"value":1},{"value":16131},{"value":2095104},{"value":2043},{"value":2657204289},{"value":18446744073709551569},{"value":2659299395},{"value":0},{"value":4294967295},{"value":2},{"value":69282117451776},{"value":3640971268},{"value":69282117451776},{"value":6129982840},{"value":8589934592},{"value":21592279046},{"value":21592279046},{"value":4412409862}],"flavor":"ARM_THREAD_STATE64","lr":{"value":8067550992},"cpsr":{"value":4096},"fp":{"value":6129982688},"sp":{"value":6129982608},"esr":{"value":1442840704,"description":" Address size fault"},"pc":{"value":8067551608},"far":{"value":0}},"frames":[{"imageOffset":4472,"symbol":"mach_msg2_trap","symbolLocation":8,"imageIndex":3},{"imageOffset":3856,"symbol":"mach_msg2_internal","symbolLocation":80,"imageIndex":3},{"imageOffset":3624,"symbol":"mach_msg_overwrite","symbolLocation":436,"imageIndex":3},{"imageOffset":3176,"symbol":"mach_msg","symbolLocation":24,"imageIndex":3},{"imageOffset":220396,"symbol":"__CFRunLoopServiceMachPort","symbolLocation":160,"imageIndex":4},{"imageOffset":211940,"symbol":"__CFRunLoopRun","symbolLocation":1208,"imageIndex":4},{"imageOffset":210504,"symbol":"CFRunLoopRunSpecific","symbolLocation":608,"imageIndex":4},{"imageOffset":181388,"symbol":"-[NSRunLoop(NSRunLoop) runMode:beforeDate:]","symbolLocation":212,"imageIndex":14},{"imageOffset":366412,"symbol":"-[NSRunLoop(NSRunLoop) runUntilDate:]","symbolLocation":64,"imageIndex":14},{"imageOffset":1633592,"symbol":"-[UIEventFetcher threadMain]","symbolLocation":420,"imageIndex":6},{"imageOffset":720352,"symbol":"__NSThread__start__","symbolLocation":732,"imageIndex":14},{"imageOffset":9428,"symbol":"_pthread_start","symbolLocation":136,"imageIndex":10},{"imageOffset":6672,"symbol":"thread_start","symbolLocation":8,"imageIndex":10}]},{"id":2878440,"frames":[{"imageOffset":6652,"symbol":"start_wqthread","symbolLocation":0,"imageIndex":10}],"threadState":{"x":[{"value":6130561024},{"value":12803},{"value":6130024448},{"value":0},{"value":409604},{"value":18446744073709551615},{"value":0},{"value":0},{"value":0},{"value":0},{"value":0},{"value":0},{"value":0},{"value":0},{"value":0},{"value":0},{"value":0},{"value":0},{"value":0},{"value":0},{"value":0},{"value":0},{"value":0},{"value":0},{"value":0},{"value":0},{"value":0},{"value":0},{"value":0}],"flavor":"ARM_THREAD_STATE64","lr":{"value":0},"cpsr":{"value":4096},"fp":{"value":0},"sp":{"value":6130561024},"esr":{"value":1442840704,"description":" Address size fault"},"pc":{"value":8646916604},"far":{"value":0}}},{"id":2878441,"frames":[{"imageOffset":6652,"symbol":"start_wqthread","symbolLocation":0,"imageIndex":10}],"threadState":{"x":[{"value":6131134464},{"value":20743},{"value":6130597888},{"value":0},{"value":409604},{"value":18446744073709551615},{"value":0},{"value":0},{"value":0},{"value":0},{"value":0},{"value":0},{"value":0},{"value":0},{"value":0},{"value":0},{"value":0},{"value":0},{"value":0},{"value":0},{"value":0},{"value":0},{"value":0},{"value":0},{"value":0},{"value":0},{"value":0},{"value":0},{"value":0}],"flavor":"ARM_THREAD_STATE64","lr":{"value":0},"cpsr":{"value":4096},"fp":{"value":0},"sp":{"value":6131134464},"esr":{"value":1442840704,"description":" Address size fault"},"pc":{"value":8646916604},"far":{"value":0}}},{"id":2878442,"frames":[{"imageOffset":6652,"symbol":"start_wqthread","symbolLocation":0,"imageIndex":10}],"threadState":{"x":[{"value":6131707904},{"value":17411},{"value":6131171328},{"value":0},{"value":409604},{"value":18446744073709551615},{"value":0},{"value":0},{"value":0},{"value":0},{"value":0},{"value":0},{"value":0},{"value":0},{"value":0},{"value":0},{"value":0},{"value":0},{"value":0},{"value":0},{"value":0},{"value":0},{"value":0},{"value":0},{"value":0},{"value":0},{"value":0},{"value":0},{"value":0}],"flavor":"ARM_THREAD_STATE64","lr":{"value":0},"cpsr":{"value":4096},"fp":{"value":0},"sp":{"value":6131707904},"esr":{"value":1442840704,"description":" Address size fault"},"pc":{"value":8646916604},"far":{"value":0}}},{"id":2878443,"name":"com.facebook.react.JavaScript","threadState":{"x":[{"value":268451845},{"value":21592279046},{"value":8589934592},{"value":93471373262848},{"value":0},{"value":93471373262848},{"value":2},{"value":4294967295},{"value":18446744073709550527},{"value":21763},{"value":0},{"value":1},{"value":21763},{"value":608592},{"value":42880953483264},{"value":0},{"value":18446744073709551569},{"value":7003640036,"symbolLocation":56,"symbol":"clock_gettime"},{"value":0},{"value":4294967295},{"value":2},{"value":93471373262848},{"value":0},{"value":93471373262848},{"value":6132276664},{"value":8589934592},{"value":21592279046},{"value":21592279046},{"value":4412409862}],"flavor":"ARM_THREAD_STATE64","lr":{"value":8067550992},"cpsr":{"value":4096},"fp":{"value":6132276512},"sp":{"value":6132276432},"esr":{"value":1442840704,"description":" Address size fault"},"pc":{"value":8067551608},"far":{"value":0}},"frames":[{"imageOffset":4472,"symbol":"mach_msg2_trap","symbolLocation":8,"imageIndex":3},{"imageOffset":3856,"symbol":"mach_msg2_internal","symbolLocation":80,"imageIndex":3},{"imageOffset":3624,"symbol":"mach_msg_overwrite","symbolLocation":436,"imageIndex":3},{"imageOffset":3176,"symbol":"mach_msg","symbolLocation":24,"imageIndex":3},{"imageOffset":220396,"symbol":"__CFRunLoopServiceMachPort","symbolLocation":160,"imageIndex":4},{"imageOffset":211940,"symbol":"__CFRunLoopRun","symbolLocation":1208,"imageIndex":4},{"imageOffset":210504,"symbol":"CFRunLoopRunSpecific","symbolLocation":608,"imageIndex":4},{"imageOffset":416964,"imageIndex":2},{"imageOffset":720352,"symbol":"__NSThread__start__","symbolLocation":732,"imageIndex":14},{"imageOffset":9428,"symbol":"_pthread_start","symbolLocation":136,"imageIndex":10},{"imageOffset":6672,"symbol":"thread_start","symbolLocation":8,"imageIndex":10}]},{"id":2878444,"name":"hades","threadState":{"x":[{"value":260},{"value":0},{"value":0},{"value":0},{"value":0},{"value":160},{"value":0},{"value":0},{"value":6132854456},{"value":0},{"value":10795549752},{"value":2},{"value":0},{"value":0},{"value":0},{"value":0},{"value":305},{"value":8385891664},{"value":0},{"value":10795549728},{"value":10795549792},{"value":6132855008},{"value":0},{"value":0},{"value":0},{"value":1},{"value":256},{"value":0},{"value":0}],"flavor":"ARM_THREAD_STATE64","lr":{"value":8646914004},"cpsr":{"value":1610616832},"fp":{"value":6132854576},"sp":{"value":6132854432},"esr":{"value":1442840704,"description":" Address size fault"},"pc":{"value":8067554076},"far":{"value":0}},"frames":[{"imageOffset":6940,"symbol":"__psynch_cvwait","symbolLocation":8,"imageIndex":3},{"imageOffset":4052,"symbol":"_pthread_cond_wait","symbolLocation":1228,"imageIndex":10},{"imageOffset":77516,"symbol":"std::__1::condition_variable::wait(std::__1::unique_lock<std::__1::mutex>&)","symbolLocation":28,"imageIndex":15},{"imageOffset":1166504,"imageIndex":1},{"imageOffset":1165916,"imageIndex":1},{"imageOffset":9428,"symbol":"_pthread_start","symbolLocation":136,"imageIndex":10},{"imageOffset":6672,"symbol":"thread_start","symbolLocation":8,"imageIndex":10}]},{"id":2878445,"name":"hermes-chrome-inspector-conn","threadState":{"x":[{"value":260},{"value":0},{"value":0},{"value":0},{"value":0},{"value":160},{"value":0},{"value":0},{"value":6133427736},{"value":0},{"value":10793391776},{"value":2},{"value":0},{"value":0},{"value":0},{"value":0},{"value":305},{"value":8385891664},{"value":0},{"value":10793391752},{"value":10793391864},{"value":6133428448},{"value":0},{"value":0},{"value":0},{"value":1},{"value":256},{"value":0},{"value":0}],"flavor":"ARM_THREAD_STATE64","lr":{"value":8646914004},"cpsr":{"value":1610616832},"fp":{"value":6133427856},"sp":{"value":6133427712},"esr":{"value":1442840704,"description":" Address size fault"},"pc":{"value":8067554076},"far":{"value":0}},"frames":[{"imageOffset":6940,"symbol":"__psynch_cvwait","symbolLocation":8,"imageIndex":3},{"imageOffset":4052,"symbol":"_pthread_cond_wait","symbolLocation":1228,"imageIndex":10},{"imageOffset":77516,"symbol":"std::__1::condition_variable::wait(std::__1::unique_lock<std::__1::mutex>&)","symbolLocation":28,"imageIndex":15},{"imageOffset":1506748,"imageIndex":2},{"imageOffset":1212792,"imageIndex":2},{"imageOffset":9428,"symbol":"_pthread_start","symbolLocation":136,"imageIndex":10},{"imageOffset":6672,"symbol":"thread_start","symbolLocation":8,"imageIndex":10}]},{"id":2878446,"name":"hermes-inspector","threadState":{"x":[{"value":260},{"value":0},{"value":0},{"value":0},{"value":0},{"value":160},{"value":0},{"value":0},{"value":6134001176},{"value":0},{"value":10793391968},{"value":2},{"value":0},{"value":0},{"value":0},{"value":0},{"value":305},{"value":8385891664},{"value":0},{"value":10793391944},{"value":10793392056},{"value":6134001888},{"value":0},{"value":0},{"value":0},{"value":1},{"value":256},{"value":0},{"value":0}],"flavor":"ARM_THREAD_STATE64","lr":{"value":8646914004},"cpsr":{"value":1610616832},"fp":{"value":6134001296},"sp":{"value":6134001152},"esr":{"value":1442840704,"description":" Address size fault"},"pc":{"value":8067554076},"far":{"value":0}},"frames":[{"imageOffset":6940,"symbol":"__psynch_cvwait","symbolLocation":8,"imageIndex":3},{"imageOffset":4052,"symbol":"_pthread_cond_wait","symbolLocation":1228,"imageIndex":10},{"imageOffset":77516,"symbol":"std::__1::condition_variable::wait(std::__1::unique_lock<std::__1::mutex>&)","symbolLocation":28,"imageIndex":15},{"imageOffset":1506748,"imageIndex":2},{"imageOffset":1212792,"imageIndex":2},{"imageOffset":9428,"symbol":"_pthread_start","symbolLocation":136,"imageIndex":10},{"imageOffset":6672,"symbol":"thread_start","symbolLocation":8,"imageIndex":10}]},{"id":2878448,"name":"com.apple.CoreMotion.MotionThread","threadState":{"x":[{"value":268451845},{"value":21592279046},{"value":8589934592},{"value":127573413593088},{"value":0},{"value":127573413593088},{"value":2},{"value":4294967295},{"value":18446744073709550527},{"value":29703},{"value":0},{"value":1},{"value":29703},{"value":3722060},{"value":1099511627776},{"value":0},{"value":18446744073709551569},{"value":7003640036,"symbolLocation":56,"symbol":"clock_gettime"},{"value":0},{"value":4294967295},{"value":2},{"value":127573413593088},{"value":0},{"value":127573413593088},{"value":6134569208},{"value":8589934592},{"value":21592279046},{"value":21592279046},{"value":4412409862}],"flavor":"ARM_THREAD_STATE64","lr":{"value":8067550992},"cpsr":{"value":4096},"fp":{"value":6134569056},"sp":{"value":6134568976},"esr":{"value":1442840704,"description":" Address size fault"},"pc":{"value":8067551608},"far":{"value":0}},"frames":[{"imageOffset":4472,"symbol":"mach_msg2_trap","symbolLocation":8,"imageIndex":3},{"imageOffset":3856,"symbol":"mach_msg2_internal","symbolLocation":80,"imageIndex":3},{"imageOffset":3624,"symbol":"mach_msg_overwrite","symbolLocation":436,"imageIndex":3},{"imageOffset":3176,"symbol":"mach_msg","symbolLocation":24,"imageIndex":3},{"imageOffset":220396,"symbol":"__CFRunLoopServiceMachPort","symbolLocation":160,"imageIndex":4},{"imageOffset":211940,"symbol":"__CFRunLoopRun","symbolLocation":1208,"imageIndex":4},{"imageOffset":210504,"symbol":"CFRunLoopRunSpecific","symbolLocation":608,"imageIndex":4},{"imageOffset":209836,"symbol":"CFRunLoopRun","symbolLocation":64,"imageIndex":4},{"imageOffset":2949024,"imageIndex":16},{"imageOffset":9428,"symbol":"_pthread_start","symbolLocation":136,"imageIndex":10},{"imageOffset":6672,"symbol":"thread_start","symbolLocation":8,"imageIndex":10}]},{"id":2878454,"name":"com.apple.NSURLConnectionLoader","threadState":{"x":[{"value":268451845},{"value":21592279046},{"value":8589934592},{"value":160610302033920},{"value":0},{"value":160610302033920},{"value":2},{"value":4294967295},{"value":18446744073709550527},{"value":37395},{"value":0},{"value":1},{"value":37395},{"value":576230},{"value":4398046511104},{"value":0},{"value":18446744073709551569},{"value":7003640036,"symbolLocation":56,"symbol":"clock_gettime"},{"value":0},{"value":4294967295},{"value":2},{"value":160610302033920},{"value":0},{"value":160610302033920},{"value":6135143768},{"value":8589934592},{"value":21592279046},{"value":21592279046},{"value":4412409862}],"flavor":"ARM_THREAD_STATE64","lr":{"value":8067550992},"cpsr":{"value":4096},"fp":{"value":6135143616},"sp":{"value":6135143536},"esr":{"value":1442840704,"description":" Address size fault"},"pc":{"value":8067551608},"far":{"value":0}},"frames":[{"imageOffset":4472,"symbol":"mach_msg2_trap","symbolLocation":8,"imageIndex":3},{"imageOffset":3856,"symbol":"mach_msg2_internal","symbolLocation":80,"imageIndex":3},{"imageOffset":3624,"symbol":"mach_msg_overwrite","symbolLocation":436,"imageIndex":3},{"imageOffset":3176,"symbol":"mach_msg","symbolLocation":24,"imageIndex":3},{"imageOffset":220396,"symbol":"__CFRunLoopServiceMachPort","symbolLocation":160,"imageIndex":4},{"imageOffset":211940,"symbol":"__CFRunLoopRun","symbolLocation":1208,"imageIndex":4},{"imageOffset":210504,"symbol":"CFRunLoopRunSpecific","symbolLocation":608,"imageIndex":4},{"imageOffset":2462616,"imageIndex":17},{"imageOffset":720352,"symbol":"__NSThread__start__","symbolLocation":732,"imageIndex":14},{"imageOffset":9428,"symbol":"_pthread_start","symbolLocation":136,"imageIndex":10},{"imageOffset":6672,"symbol":"thread_start","symbolLocation":8,"imageIndex":10}]}],
  "usedImages" : [
  {
    "source" : "P",
    "arch" : "arm64e",
    "base" : 4345135104,
    "size" : 49152,
    "uuid" : "75feca6c-f91a-30cb-8fda-b72b422fae7b",
    "path" : "\/private\/preboot\/Cryptexes\/OS\/usr\/lib\/libobjc-trampolines.dylib",
    "name" : "libobjc-trampolines.dylib"
  },
  {
    "source" : "P",
    "arch" : "arm64",
    "base" : 4350984192,
    "size" : 2998272,
    "uuid" : "d0a77ae1-fd18-3121-b65b-bf360f3f00cd",
    "path" : "\/private\/var\/containers\/Bundle\/Application\/63B03EA0-B884-4E51-93C5-2F81BA502DA8\/Joplin.app\/Frameworks\/hermes.framework\/hermes",
    "name" : "hermes"
  },
  {
    "source" : "P",
    "arch" : "arm64",
    "base" : 4339171328,
    "size" : 2883584,
    "uuid" : "b16a4957-a0ee-362a-9290-8f2f685c6c9e",
    "path" : "\/private\/var\/containers\/Bundle\/Application\/63B03EA0-B884-4E51-93C5-2F81BA502DA8\/Joplin.app\/Joplin",
    "name" : "Joplin"
  },
  {
    "source" : "P",
    "arch" : "arm64e",
    "base" : 8067547136,
    "size" : 233456,
    "uuid" : "9c958203-4390-3509-a602-f40665e29de4",
    "path" : "\/usr\/lib\/system\/libsystem_kernel.dylib",
    "name" : "libsystem_kernel.dylib"
  },
  {
    "source" : "P",
    "arch" : "arm64e",
    "base" : 6869172224,
    "size" : 5427200,
    "uuid" : "6085317a-58e1-3fad-8193-ad35001e231f",
    "path" : "\/System\/Library\/Frameworks\/CoreFoundation.framework\/CoreFoundation",
    "name" : "CoreFoundation"
  },
  {
    "source" : "P",
    "arch" : "arm64e",
    "base" : 7998812160,
    "size" : 36864,
    "uuid" : "f07e5eb6-efb0-320f-90e9-510a148db15d",
    "path" : "\/System\/Library\/PrivateFrameworks\/GraphicsServices.framework\/GraphicsServices",
    "name" : "GraphicsServices"
  },
  {
    "source" : "P",
    "arch" : "arm64e",
    "base" : 6905016320,
    "size" : 28143616,
    "uuid" : "1c84c8bc-e1e1-387c-a8d3-e25fcf28619b",
    "path" : "\/System\/Library\/PrivateFrameworks\/UIKitCore.framework\/UIKitCore",
    "name" : "UIKitCore"
  },
  {
    "source" : "P",
    "arch" : "arm64e",
    "base" : 7453573120,
    "size" : 555876,
    "uuid" : "08bb34ef-35ea-3322-a04e-247640b38309",
    "path" : "\/usr\/lib\/dyld",
    "name" : "dyld"
  },
  {
    "size" : 0,
    "source" : "A",
    "base" : 0,
    "uuid" : "00000000-0000-0000-0000-000000000000"
  },
  {
    "source" : "P",
    "arch" : "arm64e",
    "base" : 7003619328,
    "size" : 516084,
    "uuid" : "969b947e-0081-3cc6-9d4b-d74f22596ac9",
    "path" : "\/usr\/lib\/system\/libsystem_c.dylib",
    "name" : "libsystem_c.dylib"
  },
  {
    "source" : "P",
    "arch" : "arm64e",
    "base" : 8646909952,
    "size" : 49140,
    "uuid" : "28f88ffb-116c-337d-9ff3-fbb3d807f209",
    "path" : "\/usr\/lib\/system\/libsystem_pthread.dylib",
    "name" : "libsystem_pthread.dylib"
  },
  {
    "source" : "P",
    "arch" : "arm64e",
    "base" : 8645976064,
    "size" : 114688,
    "uuid" : "08ad6578-b16d-3861-930e-dc71edebb097",
    "path" : "\/usr\/lib\/libc++abi.dylib",
    "name" : "libc++abi.dylib"
  },
  {
    "source" : "P",
    "arch" : "arm64e",
    "base" : 6738608128,
    "size" : 319457,
    "uuid" : "39b6c7b0-13a7-3acf-85fd-93d8112ecfdf",
    "path" : "\/usr\/lib\/libobjc.A.dylib",
    "name" : "libobjc.A.dylib"
  },
  {
    "source" : "P",
    "arch" : "arm64e",
    "base" : 7003328512,
    "size" : 290816,
    "uuid" : "bac2a636-a40b-39b4-9431-001e94b72d9f",
    "path" : "\/usr\/lib\/system\/libdispatch.dylib",
    "name" : "libdispatch.dylib"
  },
  {
    "source" : "P",
    "arch" : "arm64e",
    "base" : 6851985408,
    "size" : 11927552,
    "uuid" : "161bb0b9-5024-3413-a011-2589ea4a0c1d",
    "path" : "\/System\/Library\/Frameworks\/Foundation.framework\/Foundation",
    "name" : "Foundation"
  },
  {
    "source" : "P",
    "arch" : "arm64e",
    "base" : 7139844096,
    "size" : 581632,
    "uuid" : "5a8104b7-4090-3a06-a2fa-da7b3eaa66dd",
    "path" : "\/usr\/lib\/libc++.1.dylib",
    "name" : "libc++.1.dylib"
  },
  {
    "source" : "P",
    "arch" : "arm64e",
    "base" : 7084060672,
    "size" : 4841472,
    "uuid" : "19756e8e-d5ce-3b3c-87b1-a21dc7240be8",
    "path" : "\/System\/Library\/Frameworks\/CoreMotion.framework\/CoreMotion",
    "name" : "CoreMotion"
  },
  {
    "source" : "P",
    "arch" : "arm64e",
    "base" : 6886850560,
    "size" : 4042752,
    "uuid" : "a83d6aab-11a8-32f9-91ef-fd29c4a9f488",
    "path" : "\/System\/Library\/Frameworks\/CFNetwork.framework\/CFNetwork",
    "name" : "CFNetwork"
  }
],
  "sharedCache" : {
  "base" : 6737674240,
  "size" : 3522723840,
  "uuid" : "b022dad7-1a9e-3b99-86cb-7a2d57546811"
},
  "vmSummary" : "ReadOnly portion of Libraries: Total=1.2G resident=0K(0%) swapped_out_or_unallocated=1.2G(100%)\nWritable regions: Total=640.6M written=0K(0%) resident=0K(0%) swapped_out=0K(0%) unallocated=640.6M(100%)\n\n                                VIRTUAL   REGION \nREGION TYPE                        SIZE    COUNT (non-coalesced) \n===========                     =======  ======= \nActivity Tracing                   256K        1 \nColorSync                          128K        4 \nCoreAnimation                      992K       29 \nFoundation                          16K        1 \nKernel Alloc Once                   32K        1 \nMALLOC                           612.6M       87 \nMALLOC guard page                  192K       12 \nSQLite page cache                  640K        5 \nSTACK GUARD                        224K       14 \nStack                             8080K       14 \nVM_ALLOCATE                       17.2M       15 \n__AUTH                            2548K      281 \n__AUTH_CONST                      33.0M      688 \n__CTF                               824        1 \n__DATA                            30.2M      687 \n__DATA_CONST                      36.5M      695 \n__DATA_DIRTY                      7828K      639 \n__FONT_DATA                          4K        1 \n__INFO_FILTER                         8        1 \n__LINKEDIT                       375.2M        4 \n__OBJC_RO                         85.6M        1 \n__OBJC_RW                         2697K        1 \n__TEXT                           821.0M      710 \ndyld private memory                272K        1 \nlibnetwork                         640K       16 \nmapped file                       45.0M       30 \nshared memory                       80K        4 \n===========                     =======  ======= \nTOTAL                              2.0G     3943 \n",
  "legacyInfo" : {
  "threadTriggered" : {
    "queue" : "com.facebook.react.BlobModuleQueue"
  }
},
  "logWritingSignature" : "4c5f718a3f8177ded28db0c0c5b81ace508db83d",
  "trialInfo" : {
  "rollouts" : [
    {
      "rolloutId" : "639124e81d92412bfb4880b3",
      "factorPackIds" : {
        "SIRI_UNDERSTANDING_TMDC" : "6396ab178061fa721c8edc69"
      },
      "deploymentId" : 250000011
    },
    {
      "rolloutId" : "60f8ddccefea4203d95cbeef",
      "factorPackIds" : {

      },
      "deploymentId" : 250000018
    }
  ],
  "experiments" : [
    {
      "treatmentId" : "cc7713ba-2d64-4df3-b646-368bf06b1aa5",
      "experimentId" : "644a477c8f139c26a6e67a9f",
      "deploymentId" : 400000002
    },
    {
      "treatmentId" : "b6f041a1-2bfe-422b-bfc4-9f9c19beb05f",
      "experimentId" : "6554710ee937b80129007632",
      "deploymentId" : 500000001
    }
  ]
}
}
leematos commented 8 months ago

@kamilczerw -- sorry for the delay. Any more details on your sync?

Images? Initial sync? Lots of data? any large images/blobs?

I think our blob handling is not as good as it should be

gaffneyd4 commented 7 months ago

@leematos I run into the same problem while syncing to a local MinIO server. I set up a development environment with an iOS simulator on the latest commit on the dev branch and can reproduce. My simulator shows this error on a sync:

JSON value '<null>' of type NSNull cannot be converted to NSString

+[RCTConvert NSString:]
    RCTConvert.m:59
__52-[RCTFileReaderModule readAsDataURL:resolve:reject:]_block_invoke
_dispatch_call_block_and_release
_dispatch_client_callout
_dispatch_lane_serial_drain
_dispatch_lane_invoke
_dispatch_root_queue_drain_deferred_wlh
_dispatch_workloop_worker_thread
_pthread_wqthread
start_wqthread

When I use a debugger and step through, it appears the app is throwing this error when it encounters a blob with a type of null:

blobId: "8DF5FBAD-BD46-4FD2-BA4E-A976BD6AE859"
name: "2_2_c0c25eb6951045fa93468114d80218cf.json"
offset: 0
size: 0
type: null

What these blobs are and where they come from, I do not know.

scaleoutsean commented 7 months ago

@gaffneyd4 that's very interesting.

I have the same problem on iPad and I just checked my S3 bucket and have no 0-sized objects. So there may be more than one variation of this bug, but at least one of them is now known.

Can you list objects in your bucket and see if the that JSON file exists?

gaffneyd4 commented 7 months ago

@scaleoutsean I have no 0 sized objects either, but I just realized that the sync lock files have this same file name format.

For example: locks/2_1_70725220b5cc4c4680ddd810aaa3dbbe.json

I wonder if my iOS client (which is quite far behind in syncing due to the crashing) is choking on long-deleted lock files.

scaleoutsean commented 7 months ago

That sounds possible, @gaffneyd4. Mine is also "behind", but it's hard to know for sure since we couldn't try the other scenario.

But being behind and trying to find an object that existed and not being able to handle could indicate a more benign error, such as not validating responses.

If you do a wget on a non-existing object, you get HTTP/1.1 404 Not Found so dealing with that that should be the first step. Instead it appears the client assumes the object does exist and then crashes when it can't convert NSNull to NSString. If they added error handling maybe it could continue past that and refresh or do whatever it normally does when it falls behind.

I had also suspected stale cache and I remember some 2-3 months ago I uninstalled Joplin and installed it again, thinking that would clear the app's cache, but that didn't help. In my case it crashes without any error only 1-2 seconds after it starts.

leematos commented 7 months ago

@gaffneyd4 -- NICE FIND!

It's been a while since I've been in the codebase but I suspect we are somewhere in this function:

https://github.com/laurent22/joplin/blob/dev/packages/lib/file-api-driver-amazon-s3.js#L236-L290

In theory the app should be treating that file as text not a blob. I wonder if minio is sending weird headers or something which is causing joplin to treat the response as a blob.

Do you happen to have a joplin log (or can you repro and check configuration -> tools -> log) I'm more familiar with debugging via that output and it will help me find the right part of the sync code to lock in on and highlight for us 👍

scaleoutsean commented 7 months ago

In theory the app should be treating that file as text not a blob.

I don't have locks/* in my bucket, but here's an example for another JSON file in Joplin bucket.

mc: <DEBUG> GET /joplin/info.json HTTP/1.1
Host: -------------
User-Agent: ---------------
Accept-Encoding: identity
Authorization: AWS4-HMAC-SHA256 Credential=FkFttyvYMp19XMhI/20240211/us-east-1/s3/aws4_request, SignedHeaders=host;x-amz-content-sha256;x-amz-date, Signature=**REDACTED**
X-Amz-Content-Sha256: e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
X-Amz-Date: 20240211T054809Z

mc: <DEBUG> HTTP/1.1 200 OK
Content-Length: 6730
Accept-Ranges: bytes
Content-Security-Policy: block-all-mixed-content
Content-Type: application/octet-stream                    <==============
Date: Sun, 11 Feb 2024 05:48:09 GMT
Etag: "ff873c5ae66d35a96b75ed2bfbc704f6"
Last-Modified: Fri, 09 Feb 2024 14:00:30 GMT
Server: MinIO
Strict-Transport-Security: max-age=31536000; includeSubDomains
Vary: Origin
Vary: Accept-Encoding
X-Amz-Id-2: e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
X-Amz-Request-Id: 17B2B8E1E483D08E
X-Content-Type-Options: nosniff
X-Minio-Compressed: s2
X-Xss-Protection: 1; mode=block

According to the AWS S3 API docs, to override values for a set of response headers in the GetObject response, you can use the following query parameters in the request: response-content-type, etc...

gaffneyd4 commented 7 months ago

logs from joplin on iphone simulator:

02-11T07:52:47: "Setting up recurrent sync with interval 300"
02-11T07:52:47: "Recurrent sync operation DISABLED!!!"
02-11T07:52:47: root: "root.biometrics: biometricsEnabled", "false"
02-11T07:52:47: root: "root.biometrics: shouldShowMainContent", "true"
02-11T07:52:47: root: "root.biometrics: this.state.sensorInfo", "{"enabled":false,"sensorsHaveChanged":false,"supportedSensors":""}"
02-11T07:52:47: root: "root.biometrics: biometricsDone", "true"
02-11T07:52:47: checkDisabledSyncItemsNotification: "No errors: Hiding notification"
02-11T07:52:47: Synchronizer: "Total resources: 9"
02-11T07:52:47: Synchronizer: "Total folders: 90"
02-11T07:52:47: Synchronizer: "Total notes: 525"
02-11T07:52:47: Synchronizer: "Operations completed: "
02-11T07:52:47: Synchronizer: "Sync: finished: Synchronisation finished [1707655965223]"
02-11T07:52:47: FileApi: "delete locks/1_2_c0c25eb6951045fa93468114d80218cf.json"
02-11T07:52:47: Synchronizer: "TaskQueue.stop: syncDownload: waiting for tasks to complete: 0"
02-11T07:52:47: Synchronizer: "TaskQueue.stop: syncDownload: Done, waited for 0"
02-11T07:52:47: FileApi: "list "
02-11T07:52:47: FileApi: "put locks/1_2_c0c25eb6951045fa93468114d80218cf.json", "null"
02-11T07:52:47: FileApi: "list "
02-11T07:52:46: FileApi: "delete locks/2_2_c0c25eb6951045fa93468114d80218cf.json"
02-11T07:52:46: FileApi: "put info.json", "null"
02-11T07:52:46: models/Setting: "Settings have been saved."
02-11T07:52:46: models/Setting: "Saving settings..."
02-11T07:52:46: FileApi: "put locks/2_2_c0c25eb6951045fa93468114d80218cf.json", "null"
02-11T07:52:46: FileApi: "list "
02-11T07:52:45: FileApi: "put locks/2_2_c0c25eb6951045fa93468114d80218cf.json", "null"
02-11T07:52:45: FileApi: "list "
02-11T07:52:45: Synchronizer: "Sync target info differs between local and remote - merging infos: ", "{"version":3,"e2ee":{"value":true,"updatedTime":1633877597928},"activeMasterKeyId":{"value":"70d6cc2f38114bb9b9fa3242f8547a73","updatedTime":1633877597928},"masterKeys":[{"id":"539cd17c085d4ba7ad21edf8598cd7de","created_time":1623512381209,"updated_time":1642358495408,"source_application":"net.cozic.joplin-desktop","encryption_method":4,"checksum":"","content":"{\"iv\":\"NR7NrEBkBgKH59suepajmA==\",\"v\":1,\"iter\":10000,\"ks\":256,\"ts\":64,\"mode\":\"ccm\",\"adata\":\"\",\"cipher\":\"aes\",\"salt\":\"O2qkYI5I/q4=\",\"ct\":\"REDACTED\"}","enabled":0,"hasBeenUsed":true},{"id":"f10aa60fed2743718bc0b91c89fcbe92","created_time":1614895857597,"updated_time":1642358494999,"source_application":"net.cozic.joplin-desktop","encryption_method":4,"checksum":"","content":"{\"iv\":\"Pez7VpeZK2bFYJhqh4JpJA==\",\"v\":1,\"iter\":10000,\"ks\":256,\"ts\":64,\"mode\":\"ccm\",\"adata\":\"\",\"cipher\":\"aes\",\"salt\":\"x2bhEWTNcWY=\",\"ct\":\"REDACTED\"}","enabled":0,"hasBeenUsed":true},{"checksum":"","encryption_method":4,"content":"{\"iv\":\"aRMAQoPwc6GPzZ2FStWvMw==\",\"v\":1,\"iter\":10000,\"ks\":256,\"ts\":64,\"mode\":\"ccm\",\"adata\":\"\",\"cipher\":\"aes\",\"salt\":\"4BdUi9FiNco=\",\"ct\":\"REDACTED\"}","created_time":1633877597912,"updated_time":1707655965903,"source_application":"net.cozic.joplin-mobile","id":"70d6cc2f38114bb9b9fa3242f8547a73","hasBeenUsed":true}],"ppk":{"value":{"id":"RS6NtiwRGmUi2tMhTcb66c","keySize":2048,"privateKey":{"encryptionMethod":4,"ciphertext":"{\"iv\":\"kUejk4iOaIH/ALKb0llUXA==\",\"v\":1,\"iter\":10000,\"ks\":256,\"ts\":64,\"mode\":\"ccm\",\"adata\":\"\",\"cipher\":\"aes\",\"salt\":\"x2bhEWTNcWY=\",\"ct\":\"REDACTED\"}"},"publicKey":"-----BEGIN RSA PUBLIC KEY-----\nMIIBCgKCAQEAyesAHWmDXNP/qpViU5wZ3+Z5CrJ4PD82PtlbsIJE/b97iisowC4z\n+ljzzk13HIEh7EnUdUsiOCDN7c6eItXOkO8bt+yWnYUJxeFtBzKD8NnV4pt6zPou\nskgQvWFugB0iB9j+2wLAoqpRi3NIJSyFKXkwAEJ+4zxm4Ku2oZvSLTxbmYbmGJTO\nNqP0/q7/lcLG5E6o5i8i/MG/HNR1pKBjyX3s7kWLAYSRWQX/bTQz0oW5c4lD17uC\nQ/ClD9ESLDJdA25KyYQqN/VwnQjeszWK6oKFnNrQXidO8//p7YWRVwBwDHQ/PRUc\n4S43nkGXf9XPXWquMRexIPUKqYROyQGvGwIDAQAB\n-----END RSA PUBLIC KEY-----","createdTime":1639062921348},"updatedTime":1639062921349},"appMinVersion":"0.00"}"
02-11T07:52:45: Synchronizer: "Sync target local info:", "{"version_":3,"masterKeys_":[{"id":"539cd17c085d4ba7ad21edf8598cd7de","created_time":1623512381209,"updated_time":1642358495408,"source_application":"net.cozic.joplin-desktop","encryption_method":4,"checksum":"","content":"{\"iv\":\"NR7NrEBkBgKH59suepajmA==\",\"v\":1,\"iter\":10000,\"ks\":256,\"ts\":64,\"mode\":\"ccm\",\"adata\":\"\",\"cipher\":\"aes\",\"salt\":\"O2qkYI5I/q4=\",\"ct\":\"REDACTED\"}","enabled":0,"hasBeenUsed":true},{"id":"f10aa60fed2743718bc0b91c89fcbe92","created_time":1614895857597,"updated_time":1642358494999,"source_application":"net.cozic.joplin-desktop","encryption_method":4,"checksum":"","content":"{\"iv\":\"Pez7VpeZK2bFYJhqh4JpJA==\",\"v\":1,\"iter\":10000,\"ks\":256,\"ts\":64,\"mode\":\"ccm\",\"adata\":\"\",\"cipher\":\"aes\",\"salt\":\"x2bhEWTNcWY=\",\"ct\":\"REDACTED\"}","enabled":0,"hasBeenUsed":true},{"checksum":"","encryption_method":4,"content":"{\"iv\":\"aRMAQoPwc6GPzZ2FStWvMw==\",\"v\":1,\"iter\":10000,\"ks\":256,\"ts\":64,\"mode\":\"ccm\",\"adata\":\"\",\"cipher\":\"aes\",\"salt\":\"4BdUi9FiNco=\",\"ct\":\"REDACTED\"}","created_time":1633877597912,"updated_time":1707655960016,"source_application":"net.cozic.joplin-mobile","id":"70d6cc2f38114bb9b9fa3242f8547a73","hasBeenUsed":true}],"appMinVersion_":"0.0.0","e2ee_":{"value":true,"updatedTime":1633877597928},"activeMasterKeyId_":{"value":"70d6cc2f38114bb9b9fa3242f8547a73","updatedTime":1633877597928},"ppk_":{"value":{"id":"RS6NtiwRGmUi2tMhTcb66c","keySize":2048,"privateKey":{"encryptionMethod":4,"ciphertext":"{\"iv\":\"kUejk4iOaIH/ALKb0llUXA==\",\"v\":1,\"iter\":10000,\"ks\":256,\"ts\":64,\"mode\":\"ccm\",\"adata\":\"\",\"cipher\":\"aes\",\"salt\":\"x2bhEWTNcWY=\",\"ct\":\"REDACTED\"}"},"publicKey":"-----BEGIN RSA PUBLIC KEY-----\nMIIBCgKCAQEAyesAHWmDXNP/qpViU5wZ3+Z5CrJ4PD82PtlbsIJE/b97iisowC4z\n+ljzzk13HIEh7EnUdUsiOCDN7c6eItXOkO8bt+yWnYUJxeFtBzKD8NnV4pt6zPou\nskgQvWFugB0iB9j+2wLAoqpRi3NIJSyFKXkwAEJ+4zxm4Ku2oZvSLTxbmYbmGJTO\nNqP0/q7/lcLG5E6o5i8i/MG/HNR1pKBjyX3s7kWLAYSRWQX/bTQz0oW5c4lD17uC\nQ/ClD9ESLDJdA25KyYQqN/VwnQjeszWK6oKFnNrQXidO8//p7YWRVwBwDHQ/PRUc\n4S43nkGXf9XPXWquMRexIPUKqYROyQGvGwIDAQAB\n-----END RSA PUBLIC KEY-----","createdTime":1639062921348},"updatedTime":1639062921349}}"
02-11T07:52:45: Synchronizer: "Sync target is already setup - checking it..."
02-11T07:52:45: Synchronizer: "Sync target remote info:", "{"version_":3,"masterKeys_":[{"id":"539cd17c085d4ba7ad21edf8598cd7de","created_time":1623512381209,"updated_time":1642358495408,"source_application":"net.cozic.joplin-desktop","encryption_method":4,"checksum":"","content":"{\"iv\":\"NR7NrEBkBgKH59suepajmA==\",\"v\":1,\"iter\":10000,\"ks\":256,\"ts\":64,\"mode\":\"ccm\",\"adata\":\"\",\"cipher\":\"aes\",\"salt\":\"O2qkYI5I/q4=\",\"ct\":\"REDACTED\"}","enabled":0,"hasBeenUsed":true},{"id":"f10aa60fed2743718bc0b91c89fcbe92","created_time":1614895857597,"updated_time":1642358494999,"source_application":"net.cozic.joplin-desktop","encryption_method":4,"checksum":"","content":"{\"iv\":\"Pez7VpeZK2bFYJhqh4JpJA==\",\"v\":1,\"iter\":10000,\"ks\":256,\"ts\":64,\"mode\":\"ccm\",\"adata\":\"\",\"cipher\":\"aes\",\"salt\":\"x2bhEWTNcWY=\",\"ct\":\"REDACTED\"}","enabled":0,"hasBeenUsed":true},{"checksum":"","encryption_method":4,"content":"{\"iv\":\"aRMAQoPwc6GPzZ2FStWvMw==\",\"v\":1,\"iter\":10000,\"ks\":256,\"ts\":64,\"mode\":\"ccm\",\"adata\":\"\",\"cipher\":\"aes\",\"salt\":\"4BdUi9FiNco=\",\"ct\":\"REDACTED\"}","created_time":1633877597912,"updated_time":1707655960016,"source_application":"net.cozic.joplin-mobile","id":"70d6cc2f38114bb9b9fa3242f8547a73","hasBeenUsed":true}],"appMinVersion_":"0.00","e2ee_":{"value":true,"updatedTime":1633877597928},"activeMasterKeyId_":{"value":"70d6cc2f38114bb9b9fa3242f8547a73","updatedTime":1633877597928},"ppk_":{"value":{"id":"RS6NtiwRGmUi2tMhTcb66c","keySize":2048,"privateKey":{"encryptionMethod":4,"ciphertext":"{\"iv\":\"kUejk4iOaIH/ALKb0llUXA==\",\"v\":1,\"iter\":10000,\"ks\":256,\"ts\":64,\"mode\":\"ccm\",\"adata\":\"\",\"cipher\":\"aes\",\"salt\":\"x2bhEWTNcWY=\",\"ct\":\"REDACTED\"}"},"publicKey":"-----BEGIN RSA PUBLIC KEY-----\nMIIBCgKCAQEAyesAHWmDXNP/qpViU5wZ3+Z5CrJ4PD82PtlbsIJE/b97iisowC4z\n+ljzzk13HIEh7EnUdUsiOCDN7c6eItXOkO8bt+yWnYUJxeFtBzKD8NnV4pt6zPou\nskgQvWFugB0iB9j+2wLAoqpRi3NIJSyFKXkwAEJ+4zxm4Ku2oZvSLTxbmYbmGJTO\nNqP0/q7/lcLG5E6o5i8i/MG/HNR1pKBjyX3s7kWLAYSRWQX/bTQz0oW5c4lD17uC\nQ/ClD9ESLDJdA25KyYQqN/VwnQjeszWK6oKFnNrQXidO8//p7YWRVwBwDHQ/PRUc\n4S43nkGXf9XPXWquMRexIPUKqYROyQGvGwIDAQAB\n-----END RSA PUBLIC KEY-----","createdTime":1639062921348},"updatedTime":1639062921349}}"
02-11T07:52:45: FileApi: "get info.json"
02-11T07:52:45: models/Folder: "updateNoteShareIds: notes to update:", "0"
02-11T07:52:45: models/Folder: "updateFolderShareIds:", "{"shareUpdateCount":0,"unshareUpdateCount":0}"
02-11T07:52:45: "ResourceService::indexNoteResources: Completed"
02-11T07:52:45: models/Setting: "Settings have been saved."
02-11T07:52:45: models/Setting: "Saving settings..."
02-11T07:52:45: Synchronizer: "Indexing resources..."
02-11T07:52:45: "ResourceService::indexNoteResources: Start"
02-11T07:52:45: Synchronizer: "Sync: starting: Starting synchronisation to target 8... supportsAccurateTimestamp = false; supportsMultiPut = false} [1707655965223]"
02-11T07:52:45: root: "root.biometrics: this.state.sensorInfo", "{"enabled":false,"sensorsHaveChanged":false,"supportedSensors":""}"
02-11T07:52:45: root: "root.biometrics: shouldShowMainContent", "true"
02-11T07:52:45: root: "root.biometrics: biometricsEnabled", "false"
02-11T07:52:45: root: "root.biometrics: biometricsDone", "true"
02-11T07:52:45: "Starting scheduled sync"
02-11T07:52:45: "Preparing scheduled sync"
02-11T07:52:44: checkDisabledSyncItemsNotification: "No errors: Hiding notification"
02-11T07:52:44: "ResourceFetcher: Auto-added resources: 0"
02-11T07:52:44: "ResourceFetcher: Auto-add resources: Mode: always"
02-11T07:52:44: "DecryptionWorker: completed decryption."
02-11T07:52:44: "Scheduling sync operation...", "1000"
02-11T07:52:44: models/Setting: "Settings have been saved."
02-11T07:52:44: models/Setting: "Saving settings..."
02-11T07:52:43: "DecryptionWorker: starting decryption..."
02-11T07:52:43: "Setting up recurrent sync with interval 300"
02-11T07:52:43: "Recurrent sync operation DISABLED!!!"
02-11T07:52:43: root: "root.biometrics: biometricsEnabled", "false"
02-11T07:52:43: root: "root.biometrics: shouldShowMainContent", "true"
02-11T07:52:43: root: "root.biometrics: this.state.sensorInfo", "{"enabled":false,"sensorsHaveChanged":false,"supportedSensors":""}"
02-11T07:52:43: root: "root.biometrics: biometricsDone", "true"
02-11T07:52:43: checkDisabledSyncItemsNotification: "No errors: Hiding notification"
02-11T07:52:43: Synchronizer: "Total notes: 525"
02-11T07:52:43: Synchronizer: "Total resources: 9"
02-11T07:52:43: Synchronizer: "Total folders: 90"
02-11T07:52:43: Synchronizer: "fetchingProcessed: 4"
02-11T07:52:43: Synchronizer: "createLocal: 2"
02-11T07:52:43: Synchronizer: "updateLocal: 1"
02-11T07:52:43: Synchronizer: "Operations completed: "
02-11T07:52:43: Synchronizer: "fetchingTotal: 4"
02-11T07:52:43: Synchronizer: "Sync: finished: Synchronisation finished [1707655959363]"
02-11T07:52:43: FileApi: "delete locks/1_2_c0c25eb6951045fa93468114d80218cf.json"
02-11T07:52:43: Synchronizer: "Sync: fetchingProcessed: Processing fetched item"
02-11T07:52:43: Synchronizer: "Sync: updateLocal: remote is more recent than local: Note: (Local 931666d4ece740078f80d44d9c8e19b2): (Remote 931666d4ece740078f80d44d9c8e19b2.md)"
02-11T07:52:42: Synchronizer: "Sync: fetchingProcessed: Processing fetched item"
02-11T07:52:42: Synchronizer: "Sync: createLocal: remote exists but local does not: (Remote c0bac18e24c74c6f99d145b5e5462ec6.md)"
02-11T07:52:42: Synchronizer: "Sync: fetchingProcessed: Processing fetched item"
02-11T07:52:42: Synchronizer: "Sync: createLocal: remote exists but local does not: (Remote 6f7471a597464375bc00e2d8b8232bd9.md)"
02-11T07:52:42: FileApi: "get info.json"
02-11T07:52:42: Synchronizer: "Sync: fetchingProcessed: Processing fetched item"
02-11T07:52:42: FileApi: "get 6f7471a597464375bc00e2d8b8232bd9.md"
02-11T07:52:42: FileApi: "get c0bac18e24c74c6f99d145b5e5462ec6.md"
02-11T07:52:42: FileApi: "get 931666d4ece740078f80d44d9c8e19b2.md"
02-11T07:52:42: Synchronizer: "Sync: fetchingTotal: Fetching delta items from sync target"
02-11T07:52:42: Synchronizer: "supportsDeltaWithItems = ", "false"
02-11T07:52:42: Synchronizer: "BasicDelta: Report: {"timestamp":1707655847740,"older":1158,"newer":4,"equal":0}"
02-11T07:52:41: FileApi: "delta "
02-11T07:52:41: Synchronizer: "TaskQueue.stop: syncDownload: waiting for tasks to complete: 0"
02-11T07:52:41: Synchronizer: "TaskQueue.stop: syncDownload: Done, waited for 0"
02-11T07:52:41: FileApi: "list "
02-11T07:52:41: FileApi: "put locks/1_2_c0c25eb6951045fa93468114d80218cf.json", "null"
02-11T07:52:41: FileApi: "list "
02-11T07:52:41: FileApi: "delete locks/2_2_c0c25eb6951045fa93468114d80218cf.json"
02-11T07:52:40: FileApi: "put info.json", "null"
02-11T07:52:40: models/Setting: "Settings have been saved."
02-11T07:52:40: models/Setting: "Saving settings..."
02-11T07:52:40: FileApi: "put locks/2_2_c0c25eb6951045fa93468114d80218cf.json", "null"
02-11T07:52:40: FileApi: "list "
02-11T07:52:40: FileApi: "put locks/2_2_c0c25eb6951045fa93468114d80218cf.json", "null"
02-11T07:52:40: FileApi: "list "
02-11T07:52:40: Synchronizer: "Sync target info differs between local and remote - merging infos: ", "{"version":3,"e2ee":{"value":true,"updatedTime":1633877597928},"activeMasterKeyId":{"value":"70d6cc2f38114bb9b9fa3242f8547a73","updatedTime":1633877597928},"masterKeys":[{"id":"539cd17c085d4ba7ad21edf8598cd7de","created_time":1623512381209,"updated_time":1642358495408,"source_application":"net.cozic.joplin-desktop","encryption_method":4,"checksum":"","content":"{\"iv\":\"NR7NrEBkBgKH59suepajmA==\",\"v\":1,\"iter\":10000,\"ks\":256,\"ts\":64,\"mode\":\"ccm\",\"adata\":\"\",\"cipher\":\"aes\",\"salt\":\"O2qkYI5I/q4=\",\"ct\":\"REDACTED\"}","enabled":0,"hasBeenUsed":true},{"id":"f10aa60fed2743718bc0b91c89fcbe92","created_time":1614895857597,"updated_time":1642358494999,"source_application":"net.cozic.joplin-desktop","encryption_method":4,"checksum":"","content":"{\"iv\":\"Pez7VpeZK2bFYJhqh4JpJA==\",\"v\":1,\"iter\":10000,\"ks\":256,\"ts\":64,\"mode\":\"ccm\",\"adata\":\"\",\"cipher\":\"aes\",\"salt\":\"x2bhEWTNcWY=\",\"ct\":\"REDACTED\"}","enabled":0,"hasBeenUsed":true},{"checksum":"","encryption_method":4,"content":"{\"iv\":\"aRMAQoPwc6GPzZ2FStWvMw==\",\"v\":1,\"iter\":10000,\"ks\":256,\"ts\":64,\"mode\":\"ccm\",\"adata\":\"\",\"cipher\":\"aes\",\"salt\":\"4BdUi9FiNco=\",\"ct\":\"REDACTED\"}","created_time":1633877597912,"updated_time":1707655960016,"source_application":"net.cozic.joplin-mobile","id":"70d6cc2f38114bb9b9fa3242f8547a73","hasBeenUsed":true}],"ppk":{"value":{"id":"RS6NtiwRGmUi2tMhTcb66c","keySize":2048,"privateKey":{"encryptionMethod":4,"ciphertext":"{\"iv\":\"kUejk4iOaIH/ALKb0llUXA==\",\"v\":1,\"iter\":10000,\"ks\":256,\"ts\":64,\"mode\":\"ccm\",\"adata\":\"\",\"cipher\":\"aes\",\"salt\":\"x2bhEWTNcWY=\",\"ct\":\"REDACTED\"}"},"publicKey":"-----BEGIN RSA PUBLIC KEY-----\nMIIBCgKCAQEAyesAHWmDXNP/qpViU5wZ3+Z5CrJ4PD82PtlbsIJE/b97iisowC4z\n+ljzzk13HIEh7EnUdUsiOCDN7c6eItXOkO8bt+yWnYUJxeFtBzKD8NnV4pt6zPou\nskgQvWFugB0iB9j+2wLAoqpRi3NIJSyFKXkwAEJ+4zxm4Ku2oZvSLTxbmYbmGJTO\nNqP0/q7/lcLG5E6o5i8i/MG/HNR1pKBjyX3s7kWLAYSRWQX/bTQz0oW5c4lD17uC\nQ/ClD9ESLDJdA25KyYQqN/VwnQjeszWK6oKFnNrQXidO8//p7YWRVwBwDHQ/PRUc\n4S43nkGXf9XPXWquMRexIPUKqYROyQGvGwIDAQAB\n-----END RSA PUBLIC KEY-----","createdTime":1639062921348},"updatedTime":1639062921349},"appMinVersion":"0.00"}"
02-11T07:52:40: Synchronizer: "Sync target local info:", "{"version_":3,"masterKeys_":[{"id":"539cd17c085d4ba7ad21edf8598cd7de","created_time":1623512381209,"updated_time":1642358495408,"source_application":"net.cozic.joplin-desktop","encryption_method":4,"checksum":"","content":"{\"iv\":\"NR7NrEBkBgKH59suepajmA==\",\"v\":1,\"iter\":10000,\"ks\":256,\"ts\":64,\"mode\":\"ccm\",\"adata\":\"\",\"cipher\":\"aes\",\"salt\":\"O2qkYI5I/q4=\",\"ct\":\"REDACTED\"}","enabled":0,"hasBeenUsed":true},{"id":"f10aa60fed2743718bc0b91c89fcbe92","created_time":1614895857597,"updated_time":1642358494999,"source_application":"net.cozic.joplin-desktop","encryption_method":4,"checksum":"","content":"{\"iv\":\"Pez7VpeZK2bFYJhqh4JpJA==\",\"v\":1,\"iter\":10000,\"ks\":256,\"ts\":64,\"mode\":\"ccm\",\"adata\":\"\",\"cipher\":\"aes\",\"salt\":\"x2bhEWTNcWY=\",\"ct\":\"REDACTED\"}","enabled":0,"hasBeenUsed":true},{"checksum":"","encryption_method":4,"content":"{\"iv\":\"aRMAQoPwc6GPzZ2FStWvMw==\",\"v\":1,\"iter\":10000,\"ks\":256,\"ts\":64,\"mode\":\"ccm\",\"adata\":\"\",\"cipher\":\"aes\",\"salt\":\"4BdUi9FiNco=\",\"ct\":\"REDACTED\"}","created_time":1633877597912,"updated_time":1707655852798,"source_application":"net.cozic.joplin-mobile","id":"70d6cc2f38114bb9b9fa3242f8547a73","hasBeenUsed":true}],"appMinVersion_":"0.0.0","e2ee_":{"value":true,"updatedTime":1633877597928},"activeMasterKeyId_":{"value":"70d6cc2f38114bb9b9fa3242f8547a73","updatedTime":1633877597928},"ppk_":{"value":{"id":"RS6NtiwRGmUi2tMhTcb66c","keySize":2048,"privateKey":{"encryptionMethod":4,"ciphertext":"{\"iv\":\"kUejk4iOaIH/ALKb0llUXA==\",\"v\":1,\"iter\":10000,\"ks\":256,\"ts\":64,\"mode\":\"ccm\",\"adata\":\"\",\"cipher\":\"aes\",\"salt\":\"x2bhEWTNcWY=\",\"ct\":\"REDACTED\"}"},"publicKey":"-----BEGIN RSA PUBLIC KEY-----\nMIIBCgKCAQEAyesAHWmDXNP/qpViU5wZ3+Z5CrJ4PD82PtlbsIJE/b97iisowC4z\n+ljzzk13HIEh7EnUdUsiOCDN7c6eItXOkO8bt+yWnYUJxeFtBzKD8NnV4pt6zPou\nskgQvWFugB0iB9j+2wLAoqpRi3NIJSyFKXkwAEJ+4zxm4Ku2oZvSLTxbmYbmGJTO\nNqP0/q7/lcLG5E6o5i8i/MG/HNR1pKBjyX3s7kWLAYSRWQX/bTQz0oW5c4lD17uC\nQ/ClD9ESLDJdA25KyYQqN/VwnQjeszWK6oKFnNrQXidO8//p7YWRVwBwDHQ/PRUc\n4S43nkGXf9XPXWquMRexIPUKqYROyQGvGwIDAQAB\n-----END RSA PUBLIC KEY-----","createdTime":1639062921348},"updatedTime":1639062921349}}"
02-11T07:52:40: Synchronizer: "Sync target is already setup - checking it..."
02-11T07:52:40: Synchronizer: "Sync target remote info:", "{"version_":3,"masterKeys_":[{"id":"539cd17c085d4ba7ad21edf8598cd7de","created_time":1623512381209,"updated_time":1642358495408,"source_application":"net.cozic.joplin-desktop","encryption_method":4,"checksum":"","content":"{\"iv\":\"NR7NrEBkBgKH59suepajmA==\",\"v\":1,\"iter\":10000,\"ks\":256,\"ts\":64,\"mode\":\"ccm\",\"adata\":\"\",\"cipher\":\"aes\",\"salt\":\"O2qkYI5I/q4=\",\"ct\":\"REDACTED\"}","enabled":0,"hasBeenUsed":true},{"id":"f10aa60fed2743718bc0b91c89fcbe92","created_time":1614895857597,"updated_time":1642358494999,"source_application":"net.cozic.joplin-desktop","encryption_method":4,"checksum":"","content":"{\"iv\":\"Pez7VpeZK2bFYJhqh4JpJA==\",\"v\":1,\"iter\":10000,\"ks\":256,\"ts\":64,\"mode\":\"ccm\",\"adata\":\"\",\"cipher\":\"aes\",\"salt\":\"x2bhEWTNcWY=\",\"ct\":\"REDACTED\"}","enabled":0,"hasBeenUsed":true},{"checksum":"","encryption_method":4,"content":"{\"iv\":\"aRMAQoPwc6GPzZ2FStWvMw==\",\"v\":1,\"iter\":10000,\"ks\":256,\"ts\":64,\"mode\":\"ccm\",\"adata\":\"\",\"cipher\":\"aes\",\"salt\":\"4BdUi9FiNco=\",\"ct\":\"REDACTED\"}","created_time":1633877597912,"updated_time":1707655879851,"source_application":"net.cozic.joplin-mobile","id":"70d6cc2f38114bb9b9fa3242f8547a73","hasBeenUsed":true}],"appMinVersion_":"0.00","e2ee_":{"value":true,"updatedTime":1633877597928},"activeMasterKeyId_":{"value":"70d6cc2f38114bb9b9fa3242f8547a73","updatedTime":1633877597928},"ppk_":{"value":{"id":"RS6NtiwRGmUi2tMhTcb66c","keySize":2048,"privateKey":{"encryptionMethod":4,"ciphertext":"{\"iv\":\"kUejk4iOaIH/ALKb0llUXA==\",\"v\":1,\"iter\":10000,\"ks\":256,\"ts\":64,\"mode\":\"ccm\",\"adata\":\"\",\"cipher\":\"aes\",\"salt\":\"x2bhEWTNcWY=\",\"ct\":\"REDACTED\"}"},"publicKey":"-----BEGIN RSA PUBLIC KEY-----\nMIIBCgKCAQEAyesAHWmDXNP/qpViU5wZ3+Z5CrJ4PD82PtlbsIJE/b97iisowC4z\n+ljzzk13HIEh7EnUdUsiOCDN7c6eItXOkO8bt+yWnYUJxeFtBzKD8NnV4pt6zPou\nskgQvWFugB0iB9j+2wLAoqpRi3NIJSyFKXkwAEJ+4zxm4Ku2oZvSLTxbmYbmGJTO\nNqP0/q7/lcLG5E6o5i8i/MG/HNR1pKBjyX3s7kWLAYSRWQX/bTQz0oW5c4lD17uC\nQ/ClD9ESLDJdA25KyYQqN/VwnQjeszWK6oKFnNrQXidO8//p7YWRVwBwDHQ/PRUc\n4S43nkGXf9XPXWquMRexIPUKqYROyQGvGwIDAQAB\n-----END RSA PUBLIC KEY-----","createdTime":1639062921348},"updatedTime":1639062921349}}"
02-11T07:52:39: FileApi: "get info.json"
02-11T07:52:39: models/Folder: "updateNoteShareIds: notes to update:", "0"
02-11T07:52:39: models/Folder: "updateFolderShareIds:", "{"shareUpdateCount":0,"unshareUpdateCount":0}"
02-11T07:52:39: "ResourceService::indexNoteResources: Completed"
02-11T07:52:39: "ResourceService::indexNoteResources: Start"
02-11T07:52:39: Synchronizer: "Indexing resources..."
02-11T07:52:39: Synchronizer: "Sync: starting: Starting synchronisation to target 8... supportsAccurateTimestamp = false; supportsMultiPut = false} [1707655959363]"
02-11T07:52:39: root: "root.biometrics: this.state.sensorInfo", "{"enabled":false,"sensorsHaveChanged":false,"supportedSensors":""}"
02-11T07:52:39: root: "root.biometrics: shouldShowMainContent", "true"
02-11T07:52:39: root: "root.biometrics: biometricsEnabled", "false"
02-11T07:52:39: root: "root.biometrics: biometricsDone", "true"
02-11T07:52:39: "Starting scheduled sync"
02-11T07:52:39: "Preparing scheduled sync"
02-11T07:52:39: "Scheduling sync operation...", "0"
02-11T07:52:38: root: "root.biometrics: this.state.sensorInfo", "{"enabled":false,"sensorsHaveChanged":false,"supportedSensors":""}"
02-11T07:52:38: root: "root.biometrics: shouldShowMainContent", "true"
02-11T07:52:38: root: "root.biometrics: biometricsEnabled", "false"
02-11T07:52:38: root: "root.biometrics: biometricsDone", "true"
02-11T07:52:36: root: "root.biometrics: this.state.sensorInfo", "{"enabled":false,"sensorsHaveChanged":false,"supportedSensors":""}"
02-11T07:52:36: root: "root.biometrics: shouldShowMainContent", "true"
02-11T07:52:36: root: "root.biometrics: biometricsEnabled", "false"
02-11T07:52:36: root: "root.biometrics: biometricsDone", "true"
gaffneyd4 commented 7 months ago

minio server output from running mc admin trace corresponding to the above iphone simulator logs time period:

2024-02-11T07:52:39.871 [200 OK] s3.GetObject redacted.fqdn/joplin/info.json?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Content-Sha256=UNSIGNED-PAYLOAD&X-Amz-Credential=joplin-127001-xr%2F20240211%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20240211T125239Z&X-Amz-Expires=3600&X-Amz-Signature=redacted&X-Amz-SignedHeaders=host&x-id=GetObject  172.21.0.1       181.359ms    ⇣  181.148267ms   ↑ 81 B ↓ 6.6 KiB
2024-02-11T07:52:40.102 [200 OK] s3.ListObjectsV2 redacted.fqdn/joplin/?delimiter=%2F&list-type=2&prefix=locks%2F  172.21.0.1       51.011ms     ⇣  50.987545ms  ↑ 182 B ↓ 267 B
2024-02-11T07:52:40.183 [200 OK] s3.PutObject redacted.fqdn/joplin/locks/2_2_c0c25eb6951045fa93468114d80218cf.json?x-id=PutObject  172.21.0.1       88.258ms     ⇣  0s         ↑ 266 B ↓ 0 B
2024-02-11T07:52:40.437 [200 OK] s3.ListObjectsV2 redacted.fqdn/joplin/?delimiter=%2F&list-type=2&prefix=locks%2F  172.21.0.1       40.302ms     ⇣  40.278246ms  ↑ 182 B ↓ 626 B
2024-02-11T07:52:40.497 [200 OK] s3.PutObject redacted.fqdn/joplin/locks/2_2_c0c25eb6951045fa93468114d80218cf.json?x-id=PutObject  172.21.0.1       326.925ms    ⇣  0s         ↑ 304 B ↓ 0 B
2024-02-11T07:52:40.879 [200 OK] s3.PutObject redacted.fqdn/joplin/info.json?x-id=PutObject  172.21.0.1       421.224ms    ⇣  0s         ↑ 6.8 KiB ↓ 0 B
2024-02-11T07:52:41.363 [204 No Content] s3.DeleteObject redacted.fqdn/joplin/locks/2_2_c0c25eb6951045fa93468114d80218cf.json?x-id=DeleteObject  172.21.0.1       245.447ms    ⇣  0s         ↑ 182 B ↓ 0 B
2024-02-11T07:52:41.641 [200 OK] s3.ListObjectsV2 redacted.fqdn/joplin/?delimiter=%2F&list-type=2&prefix=locks%2F  172.21.0.1       30.1ms       ⇣  30.075423ms  ↑ 182 B ↓ 267 B
2024-02-11T07:52:41.699 [200 OK] s3.PutObject redacted.fqdn/joplin/locks/1_2_c0c25eb6951045fa93468114d80218cf.json?x-id=PutObject  172.21.0.1       63.869ms     ⇣  0s         ↑ 266 B ↓ 0 B
2024-02-11T07:52:41.791 [200 OK] s3.ListObjectsV2 redacted.fqdn/joplin/?delimiter=%2F&list-type=2&prefix=locks%2F  172.21.0.1       30.548ms     ⇣  30.525879ms  ↑ 182 B ↓ 626 B
2024-02-11T07:52:41.888 [200 OK] s3.ListObjectsV2 redacted.fqdn/joplin/?delimiter=%2F&list-type=2&prefix=  172.21.0.1       64.303ms     ⇣  61.18888ms  ↑ 182 B ↓ 52 KiB
2024-02-11T07:52:42.357 [200 OK] s3.ListObjectsV2 redacted.fqdn/joplin/?continuation-token=ZDliNjc5NmFiNmUzNDk1Mzg5MWZhMzM1YzU5ZDdlNDYubWRbbWluaW9fY2FjaGU6djIscmV0dXJuOl0%3D&delimiter=%2F&list-type=2&prefix=  172.21.0.1       35.196ms     ⇣  34.252461ms  ↑ 182 B ↓ 8.9 KiB
2024-02-11T07:52:42.721 [200 OK] s3.GetObject redacted.fqdn/joplin/6f7471a597464375bc00e2d8b8232bd9.md?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Content-Sha256=UNSIGNED-PAYLOAD&X-Amz-Credential=joplin-127001-xr%2F20240211%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20240211T125242Z&X-Amz-Expires=3600&X-Amz-Signature=redacted&X-Amz-SignedHeaders=host&x-id=GetObject  172.21.0.1       2.054ms      ⇣  1.993235ms  ↑ 81 B ↓ 2.2 KiB
2024-02-11T07:52:42.743 [200 OK] s3.GetObject redacted.fqdn/joplin/931666d4ece740078f80d44d9c8e19b2.md?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Content-Sha256=UNSIGNED-PAYLOAD&X-Amz-Credential=joplin-127001-xr%2F20240211%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20240211T125242Z&X-Amz-Expires=3600&X-Amz-Signature=redacted&X-Amz-SignedHeaders=host&x-id=GetObject  172.21.0.1       4.026ms      ⇣  3.317482ms  ↑ 81 B ↓ 24 KiB
2024-02-11T07:52:42.740 [200 OK] s3.GetObject redacted.fqdn/joplin/c0bac18e24c74c6f99d145b5e5462ec6.md?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Content-Sha256=UNSIGNED-PAYLOAD&X-Amz-Credential=joplin-127001-xr%2F20240211%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20240211T125242Z&X-Amz-Expires=3600&X-Amz-Signature=redacted&X-Amz-SignedHeaders=host&x-id=GetObject  172.21.0.1       19.919ms     ⇣  19.807952ms  ↑ 81 B ↓ 3.8 KiB
2024-02-11T07:52:42.742 [200 OK] s3.GetObject redacted.fqdn/joplin/info.json?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Content-Sha256=UNSIGNED-PAYLOAD&X-Amz-Credential=joplin-127001-xr%2F20240211%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20240211T125242Z&X-Amz-Expires=3600&X-Amz-Signature=redacted&X-Amz-SignedHeaders=host&x-id=GetObject  172.21.0.1       285.382ms    ⇣  285.168127ms   ↑ 81 B ↓ 6.6 KiB
2024-02-11T07:52:43.346 [204 No Content] s3.DeleteObject redacted.fqdn/joplin/locks/1_2_c0c25eb6951045fa93468114d80218cf.json?x-id=DeleteObject  172.21.0.1       63.826ms     ⇣  0s         ↑ 182 B ↓ 0 B
2024-02-11T07:52:45.656 [200 OK] s3.GetObject redacted.fqdn/joplin/info.json?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Content-Sha256=UNSIGNED-PAYLOAD&X-Amz-Credential=joplin-127001-xr%2F20240211%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20240211T125245Z&X-Amz-Expires=3600&X-Amz-Signature=redacted&X-Amz-SignedHeaders=host&x-id=GetObject  172.21.0.1       241.412ms    ⇣  241.226798ms   ↑ 81 B ↓ 6.6 KiB
2024-02-11T07:52:45.987 [200 OK] s3.ListObjectsV2 redacted.fqdn/joplin/?delimiter=%2F&list-type=2&prefix=locks%2F  172.21.0.1       32.698ms     ⇣  32.673393ms  ↑ 182 B ↓ 267 B
2024-02-11T07:52:46.045 [200 OK] s3.PutObject redacted.fqdn/joplin/locks/2_2_c0c25eb6951045fa93468114d80218cf.json?x-id=PutObject  172.21.0.1       95.029ms     ⇣  0s         ↑ 266 B ↓ 0 B
2024-02-11T07:52:46.281 [200 OK] s3.ListObjectsV2 redacted.fqdn/joplin/?delimiter=%2F&list-type=2&prefix=locks%2F  172.21.0.1       32.714ms     ⇣  32.67488ms  ↑ 182 B ↓ 626 B
2024-02-11T07:52:46.343 [200 OK] s3.PutObject redacted.fqdn/joplin/locks/2_2_c0c25eb6951045fa93468114d80218cf.json?x-id=PutObject  172.21.0.1       291.118ms    ⇣  0s         ↑ 304 B ↓ 0 B
2024-02-11T07:52:46.690 [200 OK] s3.PutObject redacted.fqdn/joplin/info.json?x-id=PutObject  172.21.0.1       313.809ms    ⇣  0s         ↑ 6.8 KiB ↓ 0 B
2024-02-11T07:52:47.036 [204 No Content] s3.DeleteObject redacted.fqdn/joplin/locks/2_2_c0c25eb6951045fa93468114d80218cf.json?x-id=DeleteObject  172.21.0.1       276.298ms    ⇣  0s         ↑ 182 B ↓ 0 B
2024-02-11T07:52:47.404 [200 OK] s3.ListObjectsV2 redacted.fqdn/joplin/?delimiter=%2F&list-type=2&prefix=locks%2F  172.21.0.1       40.228ms     ⇣  40.202435ms  ↑ 182 B ↓ 267 B
2024-02-11T07:52:47.468 [200 OK] s3.PutObject redacted.fqdn/joplin/locks/1_2_c0c25eb6951045fa93468114d80218cf.json?x-id=PutObject  172.21.0.1       42.848ms     ⇣  0s         ↑ 266 B ↓ 0 B
2024-02-11T07:52:47.534 [200 OK] s3.ListObjectsV2 redacted.fqdn/joplin/?delimiter=%2F&list-type=2&prefix=locks%2F  172.21.0.1       30.543ms     ⇣  30.504885ms  ↑ 182 B ↓ 626 B
2024-02-11T07:52:47.611 [204 No Content] s3.DeleteObject redacted.fqdn/joplin/locks/1_2_c0c25eb6951045fa93468114d80218cf.json?x-id=DeleteObject  172.21.0.1       198.311ms    ⇣  0s         ↑ 182 B ↓ 0 B
gaffneyd4 commented 7 months ago

@leematos hope some of that is useful. Let me know if I can grab anything else for you.

gaffneyd4 commented 7 months ago

I keep thinking about the fact that the only platform I experience this error on is iOS. My several other computers using different builds (linux, mac arm64, mac intel) all syncing with the same MinIO server don't run into this problem. Does that give any clue about where the root cause could be?

gaffneyd4 commented 6 months ago

I've found the lines in the Synchronizer that throw the error in my simulator. Stepping over each of these lines in the debugger cause this error to throw:

JSON value '' of type NSNull cannot be converted to NSString

I'm digging further into how the lock handling code interacts with the S3 (MinIO) backend to see exactly what line we're getting stuck on.

gaffneyd4 commented 6 months ago

I think we're running into this issue when the s3 client deserializes the response (using collectBody which makes use of this streamCollector logic) from the MinIO server where native FileReader.readDataAsURL doesn't implement a null check on the blob.data.type correctly: https://github.com/facebook/react-native/issues/30378#issuecomment-731698543

That issue is unfortunately closed with no activity after 3 years.

Here is the line in the aws sdk s3 client from a (deprecated/relocated) package where we call the same FileReader method: https://github.com/aws/aws-sdk-js-v3/blob/cdc73112f552e67d0f8c5b186473a1e22c36a73f/packages/fetch-http-handler/src/stream-collector.ts#L55

The latest version of that package lives here (though that line is the same): https://github.com/smithy-lang/smithy-typescript/blob/74d1aa5b97c09cb27f8ad70188d5cb02f329b25d/packages/fetch-http-handler/src/stream-collector.ts#L55

I'm not sure how we work around the problem, as we are setting the content-type application/octet-stream when we are doing the lock puts to the s3 endpoint. The problem is the null blob type in stream from the put response returned by the s3 server.

It seems like the workaround suggestion from the issue would need to be implemented in the package used by the s3 client (https://github.com/smithy-lang/smithy-typescript/tree/main/packages/fetch-http-handler).

leematos commented 6 months ago

@gaffneyd4 HUGE! Thank you for the investigation. I wonder if we can set the lock to some sort of text and not break other syncs.

leematos commented 6 months ago

This is the s3 put code:

https://github.com/laurent22/joplin/blob/dev/packages/lib/file-api-driver-amazon-s3.js#L297-L318

In theory we can pass options, not sure what options are available to us, and/or if we want to do formatting shenanigans there to work around this.

leematos commented 6 months ago

In theory we could force some other encodings possibly: https://docs.aws.amazon.com/AWSJavaScriptSDK/v3/latest/client/s3/command/PutObjectCommand/

gaffneyd4 commented 6 months ago

Because the simulator doesn't crash, I've been able to watch it succeed in syncing up with my other installations. So the null issue doesn't appear to be occurring when doing gets or puts of my notes. If we can figure out how the fileapi is used differently by the locking code, we should be able to mimic what the normal file sync code does and circumvent the problem (only issue being this would be modifying the locking code for all the other platforms at the same time).

Edit: I see what you mean. We could try to alter the s3 driver put code so that the plain putObject call when source is not file bypasses the error somehow. Or maybe we could just avoid using the putObject call entirely.

leematos commented 6 months ago

@gaffneyd4 when you step through the locking code are you seeing what upload strategy in the s3 put driver we are taking:

https://github.com/laurent22/joplin/blob/dev/packages/lib/file-api-driver-amazon-s3.js#L306

https://github.com/laurent22/joplin/blob/dev/packages/lib/file-api-driver-amazon-s3.js#L310

leematos commented 6 months ago

If we are using uploadFileFrom it's possible our put call there needs more specifics or values shifted:

https://github.com/laurent22/joplin/blob/dev/packages/lib/file-api-driver-amazon-s3.js#L101

gaffneyd4 commented 6 months ago

@leematos Its hitting the putObject call: https://github.com/laurent22/joplin/blob/dev/packages/lib/file-api-driver-amazon-s3.js#L310

gaffneyd4 commented 6 months ago

Maybe its the difference between this (wrong) and this (right)

leematos commented 6 months ago

Those two mechanisms exist because joplin needs to handle file uploads differently than "data uploads" -- which may be a crux of the issue we are seeing here.

So, If I'm reading the lock handler code right:

https://github.com/laurent22/joplin/blob/dev/packages/lib/services/synchronizer/LockHandler.ts#L206-L208

this is the put call that will attempt to upload the lockfile.

In theory we may be able add in options to API to pass in a content-type but it's unclear why minio is not accepting this json file as is now.

Are you able to step through a lock upload and open the actual lock file? In theory it shouldn't be null and it should have some json that matches this structure: https://github.com/laurent22/joplin/blob/dev/packages/lib/services/synchronizer/LockHandler.ts#L246-L250

{
    type: LockType.Sync,
    clientType: clientType,
    clientId: clientId,
}

Your earlier messages had 02-11T07:52:47: FileApi: "put locks/1_2_c0c25eb6951045fa93468114d80218cf.json", "null"

(I don't remember if that null is joplin saying file size) but I also remember your blob being "null" and size 0: https://github.com/laurent22/joplin/issues/6923#issuecomment-1936217033

I just attempted a sync on my actual device and I'm not getting the FileApi output so I don't see the lock handling happening. I think it's because i'm not in the simulator. I'm on 12.13.10 btw.

We are inching closer here but it's still unclear if:

I did a quick google for "minio upload json" and saw https://stackoverflow.com/questions/55223401/minio-python-client-upload-bytes-directly which leads me to believe that minio's handling of our data may be the crux of the issue, and we can try to re-form it, hopefully with no ill effects for non-minio clients.

gaffneyd4 commented 6 months ago

Here's the output from the JSON.stringify(lock):

"{\"type\":2,\"clientType\":2,\"clientId\":\"c0c25eb6951045fa93468114d80218cf\"}"

When I step through, the putObject request has all those fields set. Its the blob type coming back from MinIO server that is missing a value. The data going in to the server is fine.

I can add more details to prove this, but thats what I saw when I was debugging some more last evening.

leematos commented 6 months ago

Thanks -- so, this furthers that minio is either not storing that blob right (we may need to send different content type) or it stores it and it can't read it correctly.

Can you:

I think these steps will give us the clarity on what the heck minio is doing here. I think you've done something similar in the past, but I'm not sure)

gaffneyd4 commented 6 months ago

@leematos this patch to react-native on the 0.71-stable branch prevents the crash from occurring when I run Joplin in the iOS simulator locally with it. If I PR the fix to react-native's main branch, how much work would it be to update Joplin's codebase to use latest react-native version (if they accept the patch and merge into the next release)?

leematos commented 6 months ago

Amazing work @gaffneyd4 -- Unfortunately that is above my contribution level and I think goes to @laurent22

@laurent22 👋 , Any plans to bump React Native up to latest version in the near future? Seems to fix a long running sync issue that is plaguing S3 Minio users on iOS.

scaleoutsean commented 5 months ago

@gaffneyd4 is this problem observed only with MinIO? @laurent22 what do you think about leematos' comment? If you don't want to take this route maybe we can have MinIO added to the docs as one of unsupported S3 sync targets? It's been 1.5y w/o sync for MinIO users (maybe all S3 users, I don't know) and if it becomes unsupported at least we (the users) would know we need to figure out something else rather than continue waiting.

laurent22 commented 5 months ago

Yes we'll upgrade React Native at some point, but I don't know when yet

scaleoutsean commented 5 months ago

Okay, thank you.

Fellow Joplin/iOS users who still can't sync may want to consider switching to paid service or Joplin Server (beta) on which sync works fine with iOS 16.7.6 (and possibly others) as well as with desktop clients. I deployed it yesterday.