iOS Beta, version 2024.25+

[biblicabeebli]

2024.43 is live on Testflight, please upgrade to it, please also try installing version 2.5.0 on the app store and then upgrading, as well as installing and registering.

• This build fixes the iOS 18 crash. (and also holy crap that crash)
• This build has had virtually all code libraries updated due to cascading requirements changes, random things may have changed. We need wide-spectrum testing of all features.
• Please include weekly surveys in your test studies.
  • They don't need to test anything in the survey, there is an issue where a participant could fail to correctly generate new "Scheduled Events" affecting weekly surveys.
• Please report graphical glitches and other visually-observable changes.
  • With screenshots!
  • The UI libraries were revved, and believe-it-or-not you are probably a better judge of whether something has changed
  • I'm seeing some graphics glitches in surveys, my device is set to reduced motion and dark mode and there are sometimes interactions.
• Please attempt installing this version of the app over an existing Beiwe app installation first.
• Squashed bug, please test: https://github.com/onnela-lab/beiwe-ios/issues/47

Notes so far:

• black/white bar during registration.
• see https://github.com/onnela-lab/beiwe-ios/issues/59 for discussion
• see https://github.com/onnela-lab/beiwe-backend/issues/402 for backend investigation.
• ~registration-time crash on ios 18?~

[MMel099]

I'm having crashing with this new version. Tried installing on top of the old and then uninstalling - getting crashes with both. After uninstalling, I was able to open the app and perform a few actions before it would crash. Originally tried this on iOS 17.5.1 and have since updated to the new 18.0.1. On my new OS, I am unable to open the app.

[biblicabeebli]

(perfect....)

[jprince127]

Hello! I wanted to concur with Max, I'm unable to open the app due to it crashing. The app either immediately crashes without displaying any graphics or it crashes during the screen with the beiwe logo. I'm on an iPhone with the iOS version 18.0.

[biblicabeebli]

@jprince127 @MMel099 can you try build 2024.27?

[biblicabeebli]

It is not crashing when launched in the ios 18 simulator.... Were you upgrading over an older app installation?

[biblicabeebli]

@jprince127 @MMel099 @hydawo can you try build 2024.28 and report back? I will be pushing it out shortly.

[MMel099]

@biblicabeebli Hey Eli, is 2024.28 on TestFlight? I'm just seeing 2024.27 on there right now. I just installed that one and got through registration and settings but then it started crashing.

[biblicabeebli]

Not yet, sometimes it takes a while.

[biblicabeebli]

2024.28 is now available on testflight.

[MMel099]

Just saw it come through - downloaded on top of 2024.27 and it crashed immediately. After deleting and redownloading, it let me register and navigate the settings before crashing. Now it's crashing upon start.

[hydawo]

Running iOS 18 on iPhone 15 Pro Max using latest TestFlight iOS v2.5.2 (2024.28). I'm testing on my Staging study - Hassan_Beta_Test_2_13_23 using ID j8xvmhdl.

Study is configured with:

• Daily survey at 10AM
• Relative surveys at 11AM for Days 1-15 anchored to Enrollment Date
• Absolute surveys (7 surveys scheduled for the week prior to registration)

Testing Feedback:

• Installed from Testflight -> app crashed on registration
• Launched app -> app crashed
• I did receive a push notification (despite the above crashes), but app crashed when interacting with notification. Push notification page shows resend logic kicked in for all of the past absolute survey times

Will wait for next TestFlight version to continue testing...

@MMel099 @jprince127 when the next build is released in TestFlight, can you please register to my study above. You can create a new ID for you to use.

@biblicabeebli any other testing conditions we should specifically configure the study for?

[biblicabeebli]

there's updates about what different releases are tests of in the release notes, I have to push out variations.

Old releases / their notes are still available under the previous releases menu in testflight if you want to check progress.

[biblicabeebli]

(I will make it very clear in the app release description when it [finally] works, and post here.)

[biblicabeebli]

@jprince127 @hydawo @MMel099 I've pushed out Build 2024.42 and 2024.43

• 2024.42 was the first attempt that finally got iOS 18 working again
• 2024.43 is cleanup after the fact.

I'm keeping 2024.42 up for now just in case, but I'm going to retire everything before them.

[biblicabeebli]

New bug! It looks like some action on the website regenerates absolute survey scheduled events - @hydawo registered an absolute survey, something happened, then received those one-time absolute surveys again.

This is probably unrelated to the new work, it is probably an existing bug. @biblicabeebli check when we regenerate all schedules, we probably have something regenerating all scheduled events instead of a specific type of scheduled event.

@hydawo will test a repeat of actions on the frontend to see if he can replicate it.

• Moved to https://github.com/onnela-lab/beiwe-backend/issues/407

[biblicabeebli]

@MMel099 @jprince127 @hydawo Please watch the app badge counter to make sure it is fixed.

[hydawo]

@MMel099 @jprince127 some specific tests to run:

• Daily survey at 10:15AM (WE UPDATED THIS DURING RA MEETING TODAY 11/5/24 ~4:30PM)
• Relative surveys at 11AM for Days 1-15 anchored to Enrollment Date
• Absolute surveys (7 surveys scheduled for the week prior to registration) --> these should be sent shortly after registration. Take note of 1) how many notifications you receive and 2) after taking the survey, does it repopulate in the survey queue/do you continue to get notifications?

Max What happens if you don’t take a survey?

• Test 1: don't take survey until 7 or 8pm
• Test 2: don't take survey at all for a day, pay attention to notification behavior on day after. Make sure there's only 1 copy of each daily/relative survey to take the day after you skip them

Is resend logic timer relative to individual survey/deployment time or is it "global" across all surveys (all surveys are on same timer cycle)

• Test 1: Receive weekly Survey at 10:15AM, don't take, don't open. Confirm you receive resend push at 10:45AM. Don't take, don't open.
• Receive relative Survey at 11AM, don't take, don't open.
• 11:15AM How many resend pushes do you receive?
• 11:30AM How many resend pushes do you receive?

Jenny Do pushes and survey package download occur over data as well as wifi?

• Test 1: take your phone off WiFi for a day, record behavior of notifications

Does the device communicate with the server once a survey is complete?

• For example, if I have a survey to complete daily at 9 and I have push notifications going out every 30 minutes, and I finish at 9:30. Will I get notifications for the rest of the day?
• No, device confirms that survey was received when user interacts with notification or opens app
• Test 1: interact with a survey notification, make sure survey shows in queue, do not take and see if notifications continue
• Test 2: interact with a survey notification, make sure survey shows in queue and start taking survey, do not complete, see if notifications continue

Eli @biblicabeebli IMPORTANT: If participant is in different timezone than study and notifications localize to that timezone, does resend logic check against the study timezone deployment time or the participant localized deployment time?

IMPORTANT TLDR: Is resend logic timer relative to individual survey/deployment time or is it "global" across all surveys (all surveys are on same timer cycle)

• Survey A deploys at 10AM, Survey B deploys at 10:45AM. Resend logic is set to cycle every 30 min.
• User doesn't take either survey when it comes in. At 10:30AM user should get resend push for Survey A
• User receives Survey B at 10:45AM and doesn't take
• What happens at 11AM? Do both Survey A and B send resend pushes or does Survey A resend come at 11AM and Survey B resend at 11:15AM

Won’t more frequent notifications overwhelm the notification dashboard and make it less interpretable for study teams?

• Yes, let's explore having these notifications hidden by default, but can be toggled on if needed (similar to heartbeat)

TLDR: what happens if resend logic of instance 1 of Survey A coincides with initial deployment of instance 2 of Survey A What happens if, say, you have a survey on a daily schedule, but the push notifications go out every 24 hours Would theoretically two “pings” go out at the exact same time? One for the push and one for the normal survey schedule? Or exactly one? What happens if, say, you have a survey on a daily schedule, but the resend logic was accidentally configured to go out every 26 hours Are there any possible occurrences where someone would end up filling the same survey twice?

Changing an Intervention Date

• Seems to recreate a user's entire schedule for all surveys
• Max registered, received absolute survey pushes (those set in the past prior to registration), took survey and confirmed it was no longer available. The next day, Max changed his enrollment date and received a new push for absolute survey. He took it and it cleared from queue. Survey did repeat. How many times does this repeat? Is it once for every absolute instance that was missed?
• Max also received a relative survey that he should have received this morning after resetting his intervention date. This is good and what we would want.

Done How does this affect “always available surveys?

• Test 1: Max/Hassan/Jenny took always available survey during RA meeting today (11/5/24). Survey repopulates. No notifications thus far. What are the resend notification frequencies anchored to?
• Resend logic is anchored to original deployment time of the notification

[hydawo]

@MMel099

For Yale, let's have them do basic tests: Upon registration

• Enroll, set enrollment date
• Record which surveys are initially available
• Take one copy of ALWAYS AVAILABLE and one copy of ABSOLUTE
• Record if ABSOLUTE survey repopulates, take it, count how many times it repopulates

Throughout enrollment

• Check that DAILY notification comes in every day at 10:15AM and RELATIVE notification every day at 11AM
• On some days, take DAILY survey before RELATIVE survey comes in (between 10:15AM and 11AM). Then take RELATIVE survey after it is sent at 11AM
• On some days, take both surveys in the evening
• On some days, don't take either survey

[biblicabeebli]

@MMel099 @hydawo @jprince127 I've got a server-side update in the works for testing, please keep track of items that do and don't work correctly so we can check for regressions when I next update the server.

• (tech detail - there is a known bug where absolute schedules get regenerated, working on that bug has lead to some larger rework to prevent that class of issue occurring.)

[biblicabeebli]

And let's add to the testing list - check resend behavior (and app behavior) if you clear the notfication and then proceed with various actions (opening the app, dismissing the app from the app switcher, reboot the phone...)

[MMel099]

@biblicabeebli Testing feedback TLDR: Everything behaving as normal EXCEPT one day with missing surveys

Missing surveys in question happened on Friday 11/08. I went to take the surveys around 6pm and noticed that out of the three surveys that I expected to see, only 1 was there. Always available was there while the relative and daily were not. I went in about two hours later and the relative survey appeared while the daily was still missing. My notification history confirms this - I received resend notifications until about 6pm when I opened the app for the first time.

This is on Hassan's beta study (Staging: GGcUUuyFVGvPwogTsGs4CeNR). My Beiwe id is 79awifdq

What happens if you don’t take a survey?

• Test 1: don't take survey until 7 or 8pm

Notifications continue at the expect interval until the app is opened

• Test 2: don't take survey at all for a day, pay attention to notification behavior on day after. Make sure there's only 1 copy of each daily/relative survey to take the day after you skip them

Notifications continue as expected. The next day, only one copy of each survey is available.

I will keep adding to this comment with more testing feedback

[jprince127]

Hello! I believe the survey resend behaved as expected for me.

Do pushes and survey package download occur over data as well as wifi?

• Test 1: take your phone off WiFi for a day, record behavior of notifications

I continued to get regular notifications without Wifi

Does the device communicate with the server once a survey is complete?

• Test 1: interact with a survey notification, make sure survey shows in queue, do not take and see if notifications continue
• Test 2: interact with a survey notification, make sure survey shows in queue and start taking survey, do not complete

Under both of these, I no longer had survey notifications for the rest of the day

However, I did notice there is some strange scheduling with the notifications. We had survey's scheduled to go out at 10:15AM and 11AM, and I received notifications at: 10:18, 10:54, 11, 11:30, 11:36, 12, and 12:06 for at least 3 days. There seems to be a 3 minute offset in the beginning for the first survey, then surveys come in at 36/30 minute increments.

[biblicabeebli]

(Ok. I am mostly recovered from the plague. SitRep!)

I just finished (god willing) and deployed to our staging server my various extremely delayed updates to push notification scheduling.

TLDR:

Sometimes a software component attacks a problem "in the wrong direction" - decisions made in the past fight against changes imposed in the future. In building the resend feature that you have been testing, I identified various other ~bugs, oversights, obscure-interactions-with the survey scheduling system. Ideally you see no change at all - but I can finally make simple deductions about whether an issue is sourced from the scheduling code or the resend code.

Uh.

Please let me know if things change, both good change things and bad change things.

(I will... review this thread soon... I have an inbox that is... on fire.)

[biblicabeebli]

@MMel099 https://github.com/onnela-lab/beiwe-ios/issues/69#issuecomment-2474473825

Missing surveys in question happened on Friday 11/08. I went to take the surveys around 6pm and noticed that out of the three surveys that I expected to see, only 1 was there. Always available was there while the relative and daily were not. I went in about two hours later and the relative survey appeared while the daily was still missing. My notification history confirms this - I received resend notifications until about 6pm when I opened the app for the first time.

OK - there was a daily (weekly-style schedule) survey that you never saw / were unable to take.

• Did that occur at any other time since then?
• Is that weekly survey notification visible in your notification history? can you track down and show me its presence / absence.

(Since it was a weekly schedule you may have hit a bug I have mentioned a few times. That bug should be resolved, if I can see a gap in history I have a snapshot of the old database that I can compare it to to determine if it was this bug.)

[biblicabeebli]

@jprince127

However, I did notice there is some strange scheduling with the notifications. We had survey's scheduled to go out at 10:15AM and 11AM, and I received notifications at: 10:18, 10:54, 11, 11:30, 11:36, 12, and 12:06 for at least 3 days. There seems to be a 3 minute offset in the beginning for the first survey, then surveys come in at 36/30 minute increments.

I'll work on that, I forgot about that, it's usually I forgot to zero some like a datetime microseconds value to a time comparison gets weird.

[MMel099]

Hey @biblicabeebli

It has not occurred since then. The notification is in the history and i pulled this screenshot which shows persistent resends up until 7pm at which point they stop.