Android 6 production phone not showing notifications, missing data

zagorsky commented 7 years ago

The symptoms:

The user reported that survey notifications didn't appear unless they opened the app. The report from researcher Garth Coombs was "They don't get the notifications of new surveys unless they launch Beiwe and log in", although it was probably the act of launching the app that started up the survey notifications again, not the act of logging in.
The user seems to be missing survey answers files. There were 20 or 21 surveys, and the user reported only missing 2. But the user has only 7 survey answers files recorded: 1/26, 1/27, 1/30, 1/31, 2/5, 2/7, 2/14.
The user has very little GPS and WiFi data. Between 1/25 (when the user enrolled) and 2/22 (when Maria downloaded data), there were:
1. Only 2 GPS files: one on 2/18 with 5 timestamps, accuracies all 20, and one on 2/20, with normal data.
2. Only 1 WiFi file, from 2/20, which has only two timestamps, although lots of data recorded for each timestamp
3. Scarce accelerometer data (2-5 files per day) from 1/25 to 2/17, but good accelerometer data (15-17 files per day) from 2/18 to 2/22. This may indicate that on 2/18, the user exempted Beiwe from battery optimization.

Information about the app & user:

Beiwe version 2.0 (level 10)
Android 6.0.1
Samsung Galaxy S5 (SM-G900T)
User was enrolled in study FAS study from January 25 to February 22
The user probably did NOT exempt Beiwe from battery optimization, and then sometime between 2/16 and 2/21, did exempt Beiwe from battery optimization:

"Check... 1. Is Beiwe listed under 'Apps not [battery] optimized'? 2. What permissions are enabled for Beiwe? 3. Are notifications enabled for Beiwe?" (from Josh's email to Maria on 2/15/17 with subject line "Fwd: Beiwe Bug") "I've had them check 2&3 and both are enabled." (from Maria's reply on 2/15/17) "Per the collaborator 'not all of the survey data seems to be coming through either (she said she's only missed 2-3 surveys, but we're missing ~15, including ones after she double-checked the notifications and battery-saver status and changed that).'" (from Maria's email on 2/22/17 with subject line "Beiwe missing data on production. RE: Beiwe Bug")

The audio file encryption code also crashed a lot

Here's the stack trace we got:

1487046704365
BeiweVersion:10, AndroidVersion:6.0.1, Product:kltetmo, Brand:samsung, HardwareId:qcom, Manufacturer:samsung, Model:SM-G900T, App Variant:clinical variant
Error message: An error occurred while executing doInBackground()
Error type: class java.lang.RuntimeException

Error-fill:
        org.beiwe.app.CrashHandler.writeCrashlog(CrashHandler.java:82)
        org.beiwe.app.CrashHandler.uncaughtException(CrashHandler.java:33)
        java.lang.ThreadGroup.uncaughtException(ThreadGroup.java:693)
        java.lang.ThreadGroup.uncaughtException(ThreadGroup.java:690)

Actual Error:
        java.lang.StringFactory.newStringFromChars(Native Method)
        java.lang.StringFactory.newStringFromChars(StringFactory.java:218)
        java.lang.StringFactory.newStringFromBytes(StringFactory.java:203)
        java.lang.StringFactory.newStringFromBytes(StringFactory.java:63)
        android.util.Base64.encodeToString(Base64.java:456)
        org.beiwe.app.storage.EncryptionEngine.toBase64String(EncryptionEngine.java:235)
        org.beiwe.app.storage.EncryptionEngine.encryptAES(EncryptionEngine.java:177)
        org.beiwe.app.storage.AudioFileManager.encryptAudioFile(AudioFileManager.java:37)
        org.beiwe.app.survey.AudioRecorderCommon$EncryptAudioFileTask.doInBackground(AudioRecorderCommon.java:123)
        org.beiwe.app.survey.AudioRecorderCommon$EncryptAudioFileTask.doInBackground(AudioRecorderCommon.java:1)
        android.os.AsyncTask$2.call(AsyncTask.java:295)
        java.util.concurrent.FutureTask.run(FutureTask.java:237)
        android.os.AsyncTask$SerialExecutor$1.run(AsyncTask.java:234)
        java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1113)
        java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:588)
        java.lang.Thread.run(Thread.java:818)

We got this stack trace at these times (EST):

Wednesday 1/25/2017, 10:19:50 PM
Wednesday 1/25/2017, 10:25:11 PM
Tuesday 1/31/2017, 11:21:41 PM
Thursday 2/2/2017, 11:31:30 PM
Tuesday 2/7/2017, 11:05:58 PM
Monday 2/13/2017, 11:31:44 PM

Only two other phones have seen this crash:

Eli's test phone, ID h6fflp, way back on Beiwe version 5, on May 16, 2016
One other test phone recently

Eli suspects this crash may indicate very heavy memory consumption on the phone.

zagorsky commented 7 years ago

Steps to resolve this

@mariasimoneau, Eli and I haven't been able to connect the stack trace to the other symptoms this phone is experiencing, except to hypothesize that both may be connected to very heavy memory usage on the phone in general. To figure out what's going on with Beiwe on this phone, we'd like more information:

Questions about the data

Could a researcher who's authorized to view the data for this study please answer these questions?

Did the user submit more surveys that we don't have survey_answers files for? Do the survey_timings files show that the user submitted (not just opened or answered, but submitted) more than 7 surveys? When the user submits a survey, the survey_timings file is supposed to record a line that says "User hit submit", along with the timestamp.
How many surveys did the user open, and did they submit all of them? How many times does the app_log have the phrase "opened survey [SURVEY_ID]", and do the timestamps of those instances match the timestamps of "Survey first rendered and displayed to user" in the survey_timings_files, or are they inconsistent?
Did survey notifications appear for more than about 9 surveys? When does the app_log have the phrase "Received Broadcast: Intent { act=[SURVEY_ID]..."? Every time that phrase appears, Beiwe should display the survey notification (unless Beiwe didn't have permission to show notifications). There's also no way for a user to open a survey unless the survey notification appears. And if it receives a broadcast more than once during a period when only one survey was supposed to display, then what probably happened is that the notification displayed multiple times (maybe the app was killed and started again) before the user submitted the survey.

Questions for the user directly

If the user is still enrolled in the study, would it be possible for them to answer:

Can the user check Beiwe's information in the Application Manager (under Settings -> Device -> Applications -> Application manager -> Beiwe, and give us all the information from that screen? If the amount of Storage used is very high, that might indicate that the phone is failing to upload data files.

zagorsky commented 7 years ago

Hi @mariasimoneau, in another thread you posted what I think are two more reports of missing survey notifications on Android 6 phones. One for a user in a study:

We just got another report of a user not getting notifications (Android (version 6.0.1). beiwe version 16-2.1.3). Should I have them try the 2.1.4 version? I asked the collaborator for data you requested before (below) and asked about their wifi connection. They already logged into Beiwe to see if that triggered the survey but it didn't.

And one on a phone that you were testing:

On Tuesday when I saw the missing notifications I couldn't have been sure it wasn't related to the "survey schedules not deleting" issue that I was testing as I was sending surveys every minute and then deleting them. however, i had two test phones side by side and twice one got a survey and the other did not.

I think upgrading from 2.1.3 to 2.1.4 is unlikely to solve anything, because we didn't push any code directly addressing this problem, but if you try it and it definitely makes a difference, I'll be interested! Also let me know if you get more information on either of the cases above.

onnela-lab / beiwe-android