Crash on sync - Githubissues

laurynas-biveinis commented 7 years ago

My setup is the same as in the fixed https://github.com/MobileOrg/mobileorg/issues/177. I have updated the app to 1.7.2, entered the correct encryption password, and synced. The progress bar goes through several files, stops at the end of one of them, and the app exits to the home screen. However, device logs do not seem to contain any MobileOrg crashes.

If I build "develop" branch in XCode, then I'm unable to repeat this crash in the simulator using the same data files - the app appears to work just fine there. But I don't see any 1.7.2 tag/release on the GitHub, thus I'm not sure myself if there are any code differences between AppStore 1.7.2 on the phone and develop branch in the simulator.

laurynas-biveinis commented 7 years ago

FWIW I deployed "develop" to a device, and there it crashes the same as AppStore 1.7.2

laurynas-biveinis commented 7 years ago

Here is what I see in the device log after a crash:

Jul 12 15:14:22 kastauyra-comms-6 MobileOrg(CFNetwork)[4832] <Notice>: Faulting in NSHTTPCookieStorage singleton
Jul 12 15:14:22 kastauyra-comms-6 MobileOrg(CFNetwork)[4832] <Notice>: Faulting in CFHTTPCookieStorage singleton
Jul 12 15:14:22 kastauyra-comms-6 MobileOrg(CFNetwork)[4832] <Notice>: Creating default cookie storage with default identifier
...
Jul 12 15:14:34 kastauyra-comms-6 backboardd(BaseBoard)[56] <Error>: Unable to get short BSD proc info for 4832: No such process
...
Jul 12 15:14:34 kastauyra-comms-6 backboardd(BaseBoard)[56] <Error>: Unable to get proc info for 4832: No such process
Jul 12 15:14:34 kastauyra-comms-6 ReportCrash[4843] <Notice>: Formulating report for corpse[4832] <private>
...
Jul 12 15:14:34 kastauyra-comms-6 ReportCrash(MobileCoreServices)[4843] <Notice>: notify_register_check() failed with error 1000000
Jul 12 15:14:34 kastauyra-comms-6 ReportCrash(CrashReporterSupport)[4843] <Notice>: report not saved because it is non-actionable
...
Jul 12 15:14:34 kastauyra-comms-6 SpringBoard(FrontBoard)[53] <Notice>: <FBApplicationProcess: 0x105d2bfe0; MobileOrg; pid: 4832> was killed by jetsam.
Jul 12 15:14:34 kastauyra-comms-6 SpringBoard[53] <Notice>: Process exited: <FBApplicationProcess: 0x105d2bfe0; MobileOrg; pid: -1> -> <FBApplicationProcessExitContext: 0x170a25f80; exitReason: jetsam; terminationReason: (none)>
Jul 12 15:14:34 kastauyra-comms-6 assertiond[63] <Notice>: Deleted job with label: UIKitApplication:com.mobileorg.mobileorg-Debug[0x63e5][63]

laurynas-biveinis commented 7 years ago

The org file that causes app exit is 1.1MB sized. Running the app in Instruments with Allocations enabled shows 1GB peak allocation around the time of the sync of that file, coming from -[OrgFileParser parse:]

mgmart commented 7 years ago

I have issued some pull requests which might solve the problem. You could try to build from one of my branches.

laurynas-biveinis commented 7 years ago

Thanks @mgmart, could you suggest which one out of the four pull request branches I should try first?

mgmart commented 7 years ago

Here is what I see in the device log after a crash:

Unfortunately I do not have access to the crash reports. Only @webframp can access those. The device log could contain some valuable hints.

could you suggest which one out of the four pull request branches I should try first?

I think bugfix/#185 is the only one which could help with your issue.

laurynas-biveinis commented 7 years ago

I have tried https://github.com/MobileOrg/mobileorg/pull/186, with the results exactly the same:

runs fine in Simulator
Instruments show 1GB peak heap use around the sync of the crashing file
quits on the phone

mgmart commented 7 years ago

Could you please attach the debug console output.

laurynas-biveinis commented 7 years ago

I must be doing something wrong or misunderstanding the request - the debug console stays empty throughout execution both on simulator and on a device

mgmart commented 7 years ago

Maybe only the variable-view-pane is active?

laurynas-biveinis commented 7 years ago

The XCode bottom pane is divided into two and the right one has "All Output" selector chosen, yet empty. The Product -> Scheme -> MobileOrg is set to Build Configuration: Debug if that matters

mgmart commented 7 years ago

That's strange. Will check this out this evening.

webframp commented 7 years ago

@mgmart are you thinking this fix might be best to include in 1.7.3, if found?

mgmart commented 7 years ago

'If found' might be the problem here. I've no idea what's happening.

mgmart commented 7 years ago

Could you please attach the debug console output.

I must be doing something wrong or misunderstanding the request - the debug console stays empty throughout execution both on simulator and on a device

@laurynas-biveinis, that was my bad. I'm using a private scheme with debug-breakpoints. That is the reason why nobody else than me is able to see console output. :feelsgood:

I'm investigating the possibilities to generate Org-files for testing at the moment. This could take a while. Another approach could be that I create a branch with my scheme included as public. I think we should go with the latter approach. If you like you could join us at for a more direct communication.

mgmart commented 7 years ago

I ran a test with a 5.4MB file. It peaked at 190MB of used memory. Syncing on device (iPhone 6plus) over Dropbox went fine without crash.

@laurynas-biveinis could you check what the outcome is if you are using the testfile?

laurynas-biveinis commented 7 years ago

Since you have pushed to develop branch, first I have retried my previous tests with it, with same results (no crash in simulator, large mem peak, crash on device). Then I tried @mgmart testfile - to try it, I only changed org-agenda-files value, keeping the rest of my org-mode config intact. org-mobile-push took maybe 20x longer than with my usual files. Then, the simulator (iPhone 5) did not crash, Instruments showed peak at 150MB, device (iPhone 6) crashed the same.

laurynas-biveinis commented 7 years ago

I should be available to join chat most of the European work days TZ

mgmart commented 7 years ago

I should be available to join chat most of the European work day TZ

That's a shame I will be only available outside of those.

mgmart commented 7 years ago

I only changed org-agenda-files value, keeping the rest of my org-mode config intact.

That means the test-file is the only large file?

org-mobile-push took maybe 20x longer than with my usual files.

Same here.

Then, the simulator (iPhone 5) did not crash, Instruments showed peak at 150MB, device (iPhone 6) crashed the same.

Before it peaked at 1GB, right? Regarding the crash: I think we have to wait for @webframp to issue the TestFlight release. He has then access to the crash file.

laurynas-biveinis commented 7 years ago

That means the test-file is the only large file?

Yes.

Before it peaked at 1GB, right?

Yes.

Regarding the crash: I think we have to wait for @webframp to issue the TestFlight release. He has then access to the crash file.

If it indeed crashes, why my local device log does not show it? It only said ": report not saved because it is non-actionable" - will TestFlight reveal it?

ckruse commented 7 years ago

It peaked at 1gb RAM usage? Then it is probably the OOM killer, isn't it? The iPhone 6 has 1GB of RAM.

laurynas-biveinis commented 7 years ago

My original idea too - but the test file by @mgmart peaked at 150MB yet the app was closed the same

mgmart commented 7 years ago

All the pending PRs are merged now and v1.7.3. is available. If it still crashes there might be at least the PC visible if the App was started from Xcode.

mgmart commented 7 years ago

@laurynas-biveinis do you think you could find the major differences in structure between your large file and the testfile?

laurynas-biveinis commented 7 years ago

1.7.3 did not crash on device with the test file. That's different from 1.7.2.

mgmart commented 7 years ago

But it does still crash with your large file?

laurynas-biveinis commented 7 years ago

Hard to say about the structure difference between the test file and my crashing file. Mine is shorter, has fewer outline levels (max 4 maybe), does not have code blocks, has long LOGBOOKs, only http:// hyperlinks, has org-crypt blocks, has #+TODO #+TAGS #+DRAWERS in the header, has a few clocktables.

I still haven't tested 1.7.3 properly with my own files, will update as soon as I do

laurynas-biveinis commented 7 years ago

AppStore 1.7.4 quit on my file the same as the previous versions. Should I get a crash report, if yes, what steps should I take?

mgmart commented 7 years ago

@webframp, are there any new crash logs?

webframp commented 7 years ago

@mgmart I don't see any new crash logs for 1.7.4 yet.

webframp commented 7 years ago

accidentally closed due to milestone adjustements

mgmart commented 7 years ago

AppStore 1.7.4 quit on my file the same as the previous versions. Should I get a crash report, if yes, what steps should I take?

As there seem not to be any crash logs I assume the app is not crashing. If you could please verify that by starting the app from Xcode? If Xcode does not stop execution by showing the PC, something else must happening.

mgmart commented 7 years ago

Created a test file with org-crypt entries and Logbook entries. Peek in Simulator was 164MB.

laurynas-biveinis commented 7 years ago

I don't see a 1.7.3 or 1.7.4 tag on git, thus I tried pulling develop branch into my local branch (my local branch only has a total of three changes: my Dropbox API key; my DEVELOPMENT_TEAM; and removal of apparently unused "pool" variable in OrgFileParser -(parse)). Afterwards I asked to run "pod install", which I did. Then attempts to build fail with "No such module 'SwitfyDropbox'". A bit of googling suggested I have to run "pod update". This updated Alamofire to 4.4.0 from 4.0.1 and SwitfyDropbox to 4.1.2 from 4.1.1, but the result still does not build, same error of missing module.

mgmart commented 7 years ago

I don't see a 1.7.3 or 1.7.4 tag on git

1.7.3 tag should be visible in git. It's not visible in Github. This is maybe because it's an annotated tag.

"No such module 'SwitfyDropbox'"

Are you working on MobileOrg.xcworkspace or MobileOrg.xcodeproj?

laurynas-biveinis commented 7 years ago

Ack re. tag - can I keep on testing develop or should I switch to a tag? I am working on xcodeproj

mgmart commented 7 years ago

You can safely use develop, I suppose.

Please use MobileOrg.xcworkspace

laurynas-biveinis commented 7 years ago

xcworkspace worked. The app reports itself as 1.7.3, but I don't think that's an issue. The app had the test file configuration from previous testing, so I tried syncing that first, and the app quit, this time with messages in XCode output window:

Success! User is logged into Dropbox. Message from debugger: Terminated due to memory issue

Then repeated with my own data files, same "Message from debugger: Terminated due to memory issue".

One more nit: the restarted app lists the crashing file in "Outlines". Clicking on it shows "No body text". This does not look entirely correct, an unsynced file should be e.g. grayed out in the Outlines view or similar instead of being shown as empty

mgmart commented 7 years ago

@laurynas-biveinis, there could be a memory leak somewhere. Could you try to follow this guideline to detect it?

laurynas-biveinis commented 7 years ago

@mgmart, I might have tried that already when I collected RAM usage data. IIRC, all memory was freed at the end of file parsing. I will retry.

If no leak, since the file size does not seem to be relevant for this issue, there could be a particular snippet causing the issue, I will see if I can reduce and then sanitize my data file to something useful.

laurynas-biveinis commented 6 years ago

I have changed my device from 6 to X, and the issue does not reproduce anymore - the app works with the original files as expected. Unfortunately this also means I won't look into memory use of org file parsing.

mgmart commented 6 years ago

Closed for now.

laurynas-biveinis commented 6 years ago

My joy was short lived. Now it OOMs on X too, on sync, on that same file: Feb 22 08:31:00 kastauyra-comms-X kernel[0] <Notice>: EXC_RESOURCE -> MobileOrg[3911] exceeded mem limit: ActiveHard 1400 MB (fatal) Feb 22 08:31:00 kastauyra-comms-X kernel[0] <Notice>: 119394.517 memorystatus: killing_specific_process pid 3911 [MobileOrg] (per-process-limit 10) - memorystatus_available_pages: 18967

laurynas-biveinis commented 2 years ago

Changed the device from X to 12 Pro and no longer can reproduce

MobileOrg / mobileorg

Crash on sync #197