question about "background refresh implemented for Watch complications and app (IMPORTANT: complication data will never be real-time, all of the time)"

[samartsevaas]

is that bug, what will be fix in the future or this it impossible on iphone widget and apple watch?

[paulplant]

WatchOS does not allow real-time updates for 3rd party complications. Hence the warning to make sure that people don't use the complications for treatment decisions.

It isn't a bug and it doesn't affect the iPhone widget.

I understand this is just a general question and not actually an issue with the app, right? If so, please close the issue 🙏

[samartsevaas]

WatchOS does not allow real-time updates for 3rd party complications. Hence the warning to make sure that people don't use the complications for treatment decisions.

It isn't a bug and it doesn't affect the iPhone widget.

I understand this is just a general question and not actually an issue with the app, right? If so, please close the issue 🙏

got it, thnx) what is about IPhone widget, background refresh is not working on my phone everytime. time to time i see refresh values(

[aug0211]

The complication update implemented in Bill Gestrich‘s Loop Caregiver app for Apple Watch is the most reliable that I’ve seen. I’ve tested somewhat extensively across various solutions and the way he’s doing this seems to be the most reliable.

I’m not totally certain how he’s doing it, but his code is quite clean and he tends to do a good job using the latest frameworks provided by Apple. Of course, Developer mode with widget kit restrictions disabled it requires.

I’m not able to port this over myself, but for somebody more capable like @paulplant or Johan, I thought I would share this in case it’s helpful or if there’s interest.

[paulplant]

I think you've given it away yourself.

There is no way to get faster updates than the way we are doing it, unless we focus the available updates into a shorter space of time.

If you enabled the "disable widgetkit restrictions" in the developer menu of the Watch, then the complication should update every time there is a new BG value received. It worked for me during testing.

[aug0211]

I’d love for that to be true because it’s easy enough I can do it myself at will for any app 😂 🤞

Unfortunately, it’s not what the tests I’ve done have indicated. It’s not unique to xDrip (which I love and use every day!). Another example includes Nightguard. For whatever reasons, the Loop Caregiver updates are more reliable than any third party app I’ve tested. My testing included both manually checking, observing freshness of data, and recording it for each app, something like 30x/day, as well as deploying custom versions of the complications for each app that just showed “ current time” to see how often that was updating. Less valid of a test but was interesting to do and gave me quick visibility into how recent each last update was.

Consistently, somehow, Loop Caregiver stayed the most recent (significantly so) in testing over time.

It could be that you saw fresh data right after deploying, because Apple allowed it to be prioritize perhaps?

I’ll load the latest xDrip complication up again on my watch and observe it alongside LCG in hopes that maybe it’s improved! I love the view xDrip gives and would really like to consolidate one more use case into xDrip if I could get the updates to run as frequently🤞

[JohanDegraeve]

I never heard of Loop Caregiver and I don't know exactly what it is used for but it's explained here how Loop Caregiver works: https://loopkit.github.io/loopdocs/nightscout/loop-caregiver/#prerequisites

NightScout sends Push notifications to the app. These are notifications which the user doesn't hear or see, but it wakes up the app in the background and allows the app to do some short processing, including up- and/or downloading stuff. It's 100% reliable assuming you have a working Internet off course, and that the user did not force close the app. When the app wakes up, it can do what it's supposed to do. I assume in this case it updates WatchOS? (I don't know, I'm sorry to say that I never used watchos and don't know exactly what it does even in xdrip)

Would be nice to have something similar for xDrip in follower mode. It's less battery consuming because with this mechanism it's ok that the app goes to sleep (suspended mode) until it receives a new push notification

[aug0211]

Can you highlight for me where in the link it explains the push notifications coming from NS to LCG?

It's entirely possible that I've overlooked this, but I did not recall seeing push notifications from Nightscout to LCG previously.

The APN and notifications flow from NS to Loop (not Loop Caregiver - which is a separate app from Loop). I am unaware of push notifications flowing from NS to Loop Caregiver.

Loop is a DIY automated insulin delivery app that allows users to connect to a CGM and insulin delivery device to manage insulin delivery.

Loop Caregiver is a related, but separate, DIY app which connects to Nightscout and allows remote caregivers to (eg, a parent) to send commands through Nightscout to Loop, to do things like send boluses to a kiddo who uses the Loop app, and also read in data from Nightscout to inform caregivers of BG status and general T1D management.

[aug0211]

In terms of reliability of complication updates, I grabbed 11 random samples from my Apple Watch complications (WatchOS) over the last ~3 hours.

I have 3 complications for 3 separate T1D management apps, all reading BG from Nightscout:

• Loop Caregiver ("LCG")
• Nightguard
• xDrip4iO5

Over these 11 random samples, Loop Carrgiver's complication was up to date (current to the latest BG reading) in 10 samples.

Second most reliable was Nightguard, which was up to date in 3 samples and typically ran 1-2 cycles "behind" the current BG, out of the same random 11 samples over ~3 hours.

Least reliable for Apple WatchOS complication updates was xDrip4iO5. The complication was up to date (current to the latest BG reading) in 0 of the random samples. Multiple samples ran 30 minutes or more behind or "stale". Many samples were in the 15-30 minutes (or 3-5 cycles) stale range.

Through this testing period, I did not have LoopCaregiver or Nightguard iOS apps open/running on my iPhone. The xDrip4iO5 app is open on my iPhone and running with a heartbeat (reliable updates are observed in Live Activities). I rotated xDrip4iO5 between the central rectangle complication section and the lower circular complications midway through the test in case Apple was prioritizing one of these somehow, and also in case Apple prioritized "the latest edited" complication or something tricky like this. No difference was observed after swapping spots.

I stopped testing after these 11 random samples because they (1) demonstrated a clearly observable difference and (2) reinforced data from two prior, more rigorous testing sessions spanning 24+ hours each.

Here's an image with a random sample selected, which does a nice job of summarizing the observed data across 3 apps over the same period in the same app environment. The CGM-reported BG at this time was 143 and the reading came in at 6:40 PM.

The xDrip4iO5 (center complication) sample is 30 minutes stale. The LCG (bottom middle) sample is up to date with the latest BG reading. The Nightguard (bottom right) sample is one reading behind.

[paulplant]

@aug0211 , before we continue with this, can you confirm which version of xDrip4iOS you are using?

Please tell me you're not basing this whole conversation on the initial version of the Watch Complication that was clearly documented as being released without background complication refresh?

[JohanDegraeve]

The APN and notifications flow from NS to Loop (not Loop Caregiver - which is a separate app from Loop). I am unaware of push notifications flowing from NS to Loop Caregiver.

you're right, I misunderstood the explanation.

[JohanDegraeve]

but how does it work with LGC? it's purpose is to give remote commands to Loop, I assume to someone else's Loop right? (eg kid) You're using it to get readings?

Nightguard seems to use "background fetch". Long time ago (2016) i've been testing with this. It worked during hours that I had the app open. Eg if today I use the app between 09:00 and 10:00 today (ie in the foreground), then the next day the background fetch was working fine between 09:00 and 10:00 (even if I didnt have the app open)

But things have changed a lot since then. But just I wonder. If you would delete the app (nightguard), then reinstall it, configure it. Then keep it in the background. I wonder if it will still fetch readings

[aug0211]

@aug0211 , before we continue with this, can you confirm which version of xDrip4iOS you are using?

Please tell me you're not basing this whole conversation on the initial version of the Watch Complication that was clearly documented as being released without background complication refresh?

This is based on the latest available in develop. GH shows the last commit on June 3 at 3:20 PM ET which I'm current with, version 5.2.3: b02cd82 and is current with master.

Do you have another more recent branch to recommend? Would love to test it if so.

[aug0211]

but how does it work with LGC? it's purpose is to give remote commands to Loop, I assume to someone else's Loop right? (eg kid)

You're using it to get readings?

Nightguard seems to use "background fetch". Long time ago (2016) i've been testing with this. It worked during hours that I had the app open. Eg if today I use the app between 09:00 and 10:00 today (ie in the foreground), then the next day the background fetch was working fine between 09:00 and 10:00 (even if I didnt have the app open)

But things have changed a lot since then. But just I wonder. If you would delete the app (nightguard), then reinstall it, configure it. Then keep it in the background. I wonder if it will still fetch readings

LCG pulls data from Nightscout to show caregivers. Think of it as an interface on top of NS that allows caregivers to see things like current (and past, and predicted future) BG, COB, IOB.

It also then allows caregivers to push commands such as entering carbs or sending a bolus to the looping device via NS and APNs to the Looping device.

I can try the Nightguard test if it's super helpful, though I don't know if it's the best one to look at since we've seen that it does not stay up to date super nicely?

If it helps to take Nightscout completely out of the equation (I think it's a distraction and not part of the focus), I can report that I also tested another app last night and today: Loop (not Loop Caregiver).

This complication also stayed in sync nicely on the watch when compared to xDrip4iO5 or even Nightguard - it performed more closely to Loop Caregiver.

I did not use Nightscout in this test. Loop received BGs from Dexcom Share and it was awakened every 5m via the same heartbeat device I use for xDrip4iO5.

[paulplant]

@aug0211 , before we continue with this, can you confirm which version of xDrip4iOS you are using? Please tell me you're not basing this whole conversation on the initial version of the Watch Complication that was clearly documented as being released without background complication refresh?

This is based on the latest available in develop. GH shows the last commit on June 3 at 3:20 PM ET which I'm current with, version 5.2.3: b02cd82 and is current with master.

Do you have another more recent branch to recommend? Would love to test it if so.

There's no need for a newer branch. Since 5.2.0 background refresh is implemented for the Watch complications and has worked for literally every single user without issues.

Your screenshot clearly shows that it isn't refreshing, so I think something is likely wrong with the install rather than the app as you would have installed via Xcode.

Can you open the Watch App, tap it 5 times quickly to show the debug screen and post a screenshot or photo?

[JohanDegraeve]

I can try the Nightguard test if it's super helpful, though I don't know if it's the best one to look at since we've seen that it does not stay up to date super nicely? ok no. need to test this.

[samartsevaas]

want to show u iphone widget update bug look the screens at 23:16 on different devices

iphone, xdrip4ios app on iwatch, xdrip4ios on iphone

[paulplant]

It's not a bug. Whilst iOS widgets generally update much quicker than the WatchOS widgets, they are still not always 100% (this seems especially true on the lock screen for system widgets). This is well documented even by Apple themselves.

If you want up-to-date values on the lock-screen, use the Live Activity feature.

[samartsevaas]

It's not a bug. Whilst iOS widgets generally update much quicker than the WatchOS widgets, they are still not always 100% (this seems especially true on the lock screen for system widgets). This is well documented even by Apple themselves.

If you want up-to-date values on the lock-screen, use the Live Activity feature.

ok, got it) thnx) can i close issue?

[paulplant]

We should close it but @aug0211 has opened other issues inside this one and we should probably close them first.

@aug0211 , can you show the screenshot of the Watch app debug screen? There must be a clear reason why your Watch complications are not refreshing.

[aug0211]

Image 1 - X is center complication, 1 reading behind at 110 +0 (see top right, LCG complication accurately reports that BG went to 112 +2 at 3 minutes previously, 8:40 AM.)

Image 2 - opened X on the watch, tapped 5 times. Note - data in the blue overlay updates immediately when I open on the watch (too fast for me to capture) - the BG also updates behind the blue overlay right when I open it

Image 3 - after closing X and returning to watch main screen, the complication is now up to date after the data appeared to refresh when the app opened (nothing shocking here; of course opening the app refreshes the data and the complication updates - but sharing in case it's helpful).

[aug0211]

In my previous message, I shared that I could tell data was updating very quickly when I opened the app, but it moved so fast that I could not see what changed.

Here's a video that shows me opening the app with the complication in a stale state. By scrubbing through the video slowly, we can see exactly what changes.

https://github.com/JohanDegraeve/xdripswift/assets/659845/d81c6933-ea0a-4a9a-a2dc-3585dfbc1183

[paulplant]

OK... so this is getting more confusing... you said that the watch complication wasn't updating in the background, but your first photo shows that it is (it was just 8 minutes behind)... this is normal to see.

When the Watch app is open, WatchOS allows immediate complication updates and doesn't use the forceUpdate allowance for this so no need to worry about this.

But... your debug screen is showing the "Last Heartbeat" screen... this was only included in one of my early test branches and never made it to the develop or master branches.

It was added to the code on March 21 and removed from the code on March 23 as part of my development work... so over 3 months ago and way before 5.1.0 was even released. And 5.1.0 was released without background refresh for complications at all.

So... you're definitely using some very, very out of date stuff and it seems like you probably pulled it from my development branches (outside of the xdripswift repository) in your current installation which would probably perfectly explain why things are not working.

You say that you're using the current develop branch, but as per the above your screenshots show a very out of date Watch app and therefore likely also an out of date Watch Complication target.

I would suggest deleting your existing xdripswift folder completely and recloning the project again and re-building everything. Maybe it would be worth re-building separately also the Watch app and also the Watch Complication targets to your device.

[aug0211]

you said that the watch complication wasn't updating in the background, but your first photo shows that it is (it was just 8 minutes behind)... this is normal to see.

I'm not sure if you're thinking of me or someone else perhaps.

I actually shared data and test results where the watch complication absolutely did update in the background, and I shared some observed timings and durations of updates that took place in the complication over the test that spanned about 3 hours, with all 11 samples being examples of background updates that did certainly take place.

All of my comments have been focused on the freshness of the data in the watch complication. I've highlighted two other examples that are consistently more fresh, with xDrip consistently going more stale, more often.

I'm happy to redeploy from a fresh clone of master or develop, if you think this will help, but just want to clarify first. I read back through all of my comments and they've been consistent, so if there's another misunderstanding or other data, screen shots, videos, etc that I can provide please let me know.

To state once again, the xDrip4iO5 complication in WatchOS goes out of date and becomes "stale" more often than other DIY OSS observing blood glucose. Named examples include Loop, Loop Caregiver, Nightguard. Happy to try and test a fresh clone of master (or any branch) if you think it'll help.

[aug0211]

OK... so this is getting more confusing... you said that the watch complication wasn't updating in the background, but your first photo shows that it is (it was just 8 minutes behind)... this is normal to see.

When the Watch app is open, WatchOS allows immediate complication updates and doesn't use the forceUpdate allowance for this so no need to worry about this.

But... your debug screen is showing the "Last Heartbeat" screen... this was only included in one of my early test branches and never made it to the develop or master branches.

It was added to the code on March 21 and removed from the code on March 23 as part of my development work... so over 3 months ago and way before 5.1.0 was even released. And 5.1.0 was released without background refresh for complications at all.

So... you're definitely using some very, very out of date stuff and it seems like you probably pulled it from my development branches (outside of the xdripswift repository) in your current installation which would probably perfectly explain why things are not working.

You say that you're using the current develop branch, but as per the above your screenshots show a very out of date Watch app and therefore likely also an out of date Watch Complication target.

I would suggest deleting your existing xdripswift folder completely and recloning the project again and re-building everything. Maybe it would be worth re-building separately also the Watch app and also the Watch Complication targets to your device.

I looked a bit more after reading your message.

Can you double check line 347 in the current, latest versions of WatchStateModel.swift of both master and develop?

I think this may be doing what you said was removed. But I see it in current develop and master.

debugString += "\nLast hearbeat: \(timeStampOfLastHeartBeat.formatted(date: .omitted, time: .standard))"

I went ahead and took the time to re-clone and redeploy as you recommended. What I see on my watch matches what I'd expect to see based on the code I mentioned above.

I'm confident I'm running this code as there is even a typo in it ("hearbeat") which matches what I showed in my earlier video, what I see in the latest master branch, and what I observe on my watch still after redeploying. All 3 of these point to it being consistent. I have 0 code modifications here.

Here's where I'm seeing this (and of course, locally on my dev machine as well) - see line 347:

https://github.com/JohanDegraeve/xdripswift/blob/develop/xDrip%20Watch%20App/DataModels/WatchStateModel.swift

https://github.com/JohanDegraeve/xdripswift/blob/master/xDrip%20Watch%20App/DataModels/WatchStateModel.swift

[paulplant]

@aug0211 , my apologies... you're right.

I got this mixed up. This was something I added, removed immediately, but then re-added at a later date. The app watch I was checking your screenshot against was the one that was out of date... not yours. Sorry!

So I've been thinking a bit more about this today and I can only think of one scenario that would cause what you are explaining... bearing in mind that there are only really a certain amount of ways to make the complications refresh. I think I might have an explanation that makes sense.

I checked the documentation for LCG (I haven't had time to check the code) and it clearly states that Developer Mode -> WidgetKit Developer Mode should be enabled or you will only get updates every 15-20 minutes (which is perfectly correct).

Now... there are only a few ways that you can update a complication context:

1. by sending the context data to the Watch whilst the watch app is in the foreground... this will then immediately update the complication so when you go back to the watch home screen, the complication will be "current".

2. by sending the context data to the Watch whilst the watch app is in the background (or closed). This will not send anything to the Watch. It will keep it queued on the iOS device and then send it when it decides it's a good time to do so without causing battery drains. This is what will probably happen most of the time. This generally means that the complication will get updated 3-4 times an hour.

If the user is not looking at their Watch at all, then it might decide to only update every 30-60 minutes. If the user keeps glancing at their Watch, the complications will get a slightly higher priority. If the user is opening the Watch app (that is linked to the complication), then it will also get and even higher priority (which is ironic/useless to us because by opening the app we would have updated the complication data anyway).

3. we can "manually force" a complication update by sending the context data using the .transferUserInfo() method to the Watch connectivity session. This can be done a maximum of 50 times per day. This will "usually" trigger a (reasonably quick) update of the complication (usually there is a delay). Once these 50 times are used, then we default back to (2) which might update in 10 minutes or it might update in 60 minutes, usually closer to the latter.

Now... if you enable the WidgetKit Developer Mode setting on your Watch, then you will get unlimited transferUserInfo() availability... it'll use (slightly) more battery, but if you send the context data using (3) every 5 minutes, it will update the complication every 5 minutes which is how most people would want the complications to work all the time.

If you try and do that every 5 minutes for a normal Testflight or App Store install, then WidgetKit Developer Mode is not available. This means that you would use up your 50 transfers in just a few hours (potentially a lot less) and therefore by around mid-morning, you would automatically default back to (2) which would be really slow updates for all users who didn't install with Xcode and they would have a very bad experience until bed-time.

This is obviously not acceptable for xDrip4iOS as probably 99.5% of our users install via Testflight or the App Store (Shuggah) so we try to respect the correct way of doing things and cannot count on the users enabling WidgetKit Developer Mode... we try and run (2) whenever we get a new BG value, but we also run (3) every 17-20 minutes.... (assuming 15-16 hours of effective Watch use by day divided by 50 transfers gives us this frequency).

This means that we get the best and most reliable update for all users. However, it also means that enabling the "WidgetKit Developer Mode" won't actually do much for xDrip4iOS as (3) is only used every 17-20 minutes anyway.

Now, seeing as how LCG documentation specifically tells the users that they should always enable WidgetKit Developer Mode, I guess that they are purposefully using (3) as their main update method... and seeing as how probably 99% of the users all install through Xcode, then they can take advantage of this little workaround to force regular updates even if it would cause a very noticeable detrimental effect to any Testflight users (of which there probably aren't many at all) and App Store users (of which there are none).

Please read carefully through this and let me know if it makes sense to you. The more I think about it, the more it would make sense to explain what you are seeing.

[aug0211]

Hi Paul, thanks for taking the time to write all this up and help me understand better. This theory sounds very logical!

One odd thing, I cannot find any calls to transferUserInfo in the Loop Caregiver code. Fascinating, because I was sure your theory must be it. But it appears that LCG is not using this mechanism to trigger updates.

I found where the XDrip code is implementing the logic you explained above (nicely commented, lines 139-163 inside of sendStateToWatch() inside of WatchManager.swift).

I think the simplest way for me to test this theory is to override the definition of the updateWatchApp() function in the same spot (line 170) such that it always use "true" (even when false is passed in) as the forceComplicationUpdate argument passed to processWatchState. I expect this to force the more aggressive triggering for all readings for users with developer mode enabled, based on the || logic .

Now, there's one other thing that I found interesting and perhaps worth mentioning - especially since LCG does not appear to use the transferUserInfo approach (maybe there's another approach that is interesting?). I noticed that the Loop Caregiver watch app continues to receive updates for BG when the paired phone is turned off. XDrip on Watch seems to stop receiving readings when the paired phone is off. I tried this by... well, turning the paired phone off, and watching both apps (opening back and forth) on my watch, for two reading cycles. LCG updated for both, and XDrip did not. This may be totally irrelevant and not at all helpful, but it felt somewhat interesting and thought I'd share.

Anyway, I'll test with the forceComplicationUpdate: true (and reducing the defaultForceComplicationRefreshTimeInMinutes: Int = 17 down to 1 minute for good measure) and report back. Hopefully this is it!