search

opentrace-community / opentrace-ios

OpenTrace iOS app. Reference implementation of the BlueTrace protocol.
https://bluetrace.io
GNU General Public License v3.0
432 stars 135 forks source link

Overcoming the iOS background Bluetooth limitations #4

Open vitalii-tym opened 4 years ago

vitalii-tym commented 4 years ago

Problem: It is almost impossible to keep the app running, the app will be suspended soon after home screen is locked (sooner or later, depending on RAM available, energy consumption and a number of other factors). In real world a user can keep their phone unlocked and app running only in ideal situation while at home or in office. This is almost impossible to do while commuting or shopping.

Possible solution: while app is suspended there is a way to delegate the tracking job to iOS itself by setting a BLE beacon monitoring using CoreLocation services. Once iOS detects a beacon it would bring the app from suspended/inactive state into background and will give it a small amount of time to do its job.

The idea is to make those copies of the app which are in foreground to act as BLE beacons from time to time. So that iOS would wake up those copies of the app, which are suspended as soon as they get into beacon's region.

This functionality can be added independently, main bluetooth communication functionality doesn't need to be changed for this to work.

By knowing the UUID any BLE-enabled device will be able to pretend to be the app and "wake" other copies of the app. I don't expect this to be a security problem. From other side, it makes it possible to install beacons in public places and transport if needed.

While implementing beacon monitoring for my client I have found that beacon monitoring is not 100% reliable. For example when phone is in "Power saving mode", the iOS would wake the app only if user lights on the screen (by clicking the power button or turning the phone in "looking at screen" position). But the majority of time it was working, including the cases when app was force quit by user.

Please let me know if you consider this to be a valid solution. I can try to find some spare time for a PR.

ramsestom commented 4 years ago

Another solution would be to use background modes (https://developer.apple.com/library/archive/documentation/General/Reference/InfoPlistKeyReference/Articles/iPhoneOSKeys.html#//apple_ref/doc/plist/info/UIBackgroundModes) to allow the app to work continuously in background. The advantage over the beacon monitoring solution (that would wake your app for only ~10s when a beacon's region enter or exit event is triggered) is that it would still allow iOS devices to advertise (and not just listen) when screen is locked so you won't miss a contact between two iOS devices that would be screen locked for example. Background modes is usually the solution used by music players or GPS runing or navigation apps to continue to work even when the screen is locked or the app is backgrounded. You have to justify this usage for apple not to reject your app though but I am sure that for an app intended to fight the COVID-19 epidemic they should be rather conciliatory

vitalii-tym commented 4 years ago

Had some time to take a look at the code again. The project indeed has Background Modes already enabled specifically for BLE. Also the app has opted in for the State Preservation and Restoration feature (which makes it possible to bring a completely closed app into background) as described here.

I can see that the code in centralManager:willRestoreState: is commented and peripheralManager:willRestoreState: is also empty. Might be unfinished work or maybe something went wrong there. Yet the doc states:

Important: For apps that opt in to the state preservation and restoration feature of Core Bluetooth, these are the first methods (centralManager:willRestoreState: and peripheralManager:willRestoreState:) invoked when your app is relaunched into the background.

So maybe something went wrong. It would be good to hear from developers what was the exact problem they faced and if they need help from the community with it.

UPD: There is also a small trick that might give extra time to finish scanning cycle (which is currently configured to 10 secs): after app has been brought into background a combination of UIApplication.shared.beginBackgroundTask and UIApplication.shared.endBackgroundTask may give several extra seconds, in some cases it may extend background time to a few minutes (when the battery level is good enough and app's energy consumption is moderate).

BTW, heres what Apple posted today: https://www.apple.com/newsroom/2020/04/apple-and-google-partner-on-covid-19-contact-tracing-technology/ They have preliminary specs linked there. Worth checking.

apecodingman commented 4 years ago

UIApplication.shared.isIdleTimerDisabled = true: this is the code found in AppDelegate, it seems the app is required to stay in the foreground all the time for some reason. And there's also a screen named PogoInstructionsViewController requires the app needs to be kept open to work. And the app will be covered with a dark layer when face down or upside down to save the power. So really curious about whether this app can work in the background or not.

tomacco commented 4 years ago

I've been working in the iBeacon world for a while. We (my current company and I) are currently designing a solution similar to the one proposed by @vitalii-tym and will like to add info that could be useful.

1 - When in background, iOS won't let the app transmit iBeacon packets. When you move from foreground to background, the OS will allow you to keep transmitting for 3 minutes (max if you explicitly request an extension). After waking up from a location event (iBeacon, significant location movement, geofence event etc), iOS won't let you emit as iBeacon unless you pass to foreground.

2 - From our observations (please correct me if I am wrong) the BLE scans using CoreBluetooth in background do not have enough data in order to correctly identify the emitter. How are you guys solving this issue? Do you have meaningful detections in background or only in foreground?

3 - Currently we want to use iBeacon as a means to allow people around Android devices to receive a notification so they open the application for a while. This is of course a challenge itself as you do not want to spam people with messages, so for this part we use a 3rd party provider to manage the content and cadence of the message.

Alternative Approach: iBeacon Only

Limitations:

So it is possible to fix let's say 15 UUIDs that are always the same, and use the major and minor (4 bytes in total) to assign the ephemeral user id that will be transmitted over the air. That is 64.434 million (15 x (2^32- 1)) possibilities. It is quite low compared with the uniqueness of an UUID, but still enough to map all mobile devices on earth without collisions.

This approach will allow to detect Android emitted ids in iOS automatically in the background with no restrictions, leaving us only with the problem of not being able to emit iBeacon in the background in iOS.

Hope this can be useful, and please let me know your input regarding this and the question above.

jasonbay commented 4 years ago

Background modes is usually the solution used by music players or GPS runing or navigation apps to continue to work even when the screen is locked or the app is backgrounded. You have to justify this usage for apple not to reject your app though but I am sure that for an app intended to fight the COVID-19 epidemic they should be rather conciliatory

We did consider this, but as a government app, it seemed a little too "cute". I have discussed this with a number of governments, but seeing as Apple (and Google) had hinted at a solution, we never really got around to exploring it. It might be worth trying though.

jasonbay commented 4 years ago

UIApplication.shared.isIdleTimerDisabled = true: this is the code found in AppDelegate, it seems the app is required to stay in the foreground all the time for some reason. And there's also a screen named PogoInstructionsViewController requires the app needs to be kept open to work. And the app will be covered with a dark layer when face down or upside down to save the power. So really curious about whether this app can work in the background or not.

Nope. What cannot be solved technically, we attempted to solve through user education. Hence, PoGo mode.

vitalii-tym commented 4 years ago

Here's the exact reason why the app is unable to work correctly while in the background with the current design:

Screenshot 2020-04-15 14 08 40

Bringing the app into background using CoreLocation region monitoring won't help in this case. There is a possible workaround: CoreLocation could issue a local notification saying "Hey, we have noticed you've got people around you. Touch here to open the OpenTrace app". But this will have only limited effect because people are mostly ignoring notifications.

vishnukanth commented 4 years ago

Hello Vitalii, Thanks for sharing your thoughts.

We can run a player using AVAudioSession with volume 0 (which can be a silent player). We will start this player once the app enters background and we will stop the player on didBecomeActive or enterforeground.

We can keep the app running in background by playing the silent audio periodically.

Please let me know your thoughts on this

Regards Vishnukanth Alaparthi

vitalii-tym commented 4 years ago

Hello, @vishnukanth . These are all good suggestions, but the problem is not in how to make the app work in background any more. The app already works in background mode. The problem is in the way Bluetooth behaves while in background. Sorry that the issue description looks misleading. Will update it in a moment.

vitalii-tym commented 4 years ago

Played with the BLE interaction a little more and found two interesting things:

• Firstly, if we have 2 phones and on one the app works in background mode while on another it is in foreground - both apps are still capable to find each other and exchange tempIDs (haven't verified if the rest of info is full or not, but the tempIDs are the most important thing I suppose). So even if some of users don't keep the app in PoGo mode, there are still chances for the app to log connections with other people (iPhone users only?) who do have their apps in this mode. This already works in the current version.

• Second, I have found a way to keep 2 iPhones exchanging tempIDs for a very long time while they both in background. Theoretically this will work only for phones which have already found each other. I'll stop in details about this below.

After reading through lots of stackoverflow answers, I've noticed one stating that central.connect(peripheral) would stay pending until the peripheral is connected to again. The doc for this method indeed says

Attempts to connect to a peripheral don’t time out. To explicitly cancel a pending connection to a peripheral, call the cancelPeripheralConnection(_:) method

Which means that calling central.connect(peripheral) while peripheral is nowhere around to be seen, would stay pending, and iOS would trigger Bluetooth-related methods as soon as peripheral is seen again.

So, I did the following:

This still doesn't fully solve the problem of inability to scan and find between 2 newly seen devices when both of apps are backgrounded. But it allows to keep tracking 2 people whose phones already found themselves in order to determine how long they were in contact with each other. Please, let me know if you are interested in a PR.

batthis commented 4 years ago

I believe iOS 13.5 beta 3 provides the Apple-Google APIs for contact tracing allowing Bluetooth apps like this to run in the background.

lorsk commented 4 years ago

Played with the BLE interaction a little more and found two interesting things:

• Firstly, if we have 2 phones and on one the app works in background mode while on another it is in foreground - both apps are still capable to find each other and exchange tempIDs (haven't verified if the rest of info is full or not, but the tempIDs are the most important thing I suppose). So even if some of users don't keep the app in PoGo mode, there are still chances for the app to log connections with other people (iPhone users only?) who do have their apps in this mode. This already works in the current version.

• Second, I have found a way to keep 2 iPhones exchanging tempIDs for a very long time while they both in background. Theoretically this will work only for phones which have already found each other. I'll stop in details about this below.

After reading through lots of stackoverflow answers, I've noticed one stating that central.connect(peripheral) would stay pending until the peripheral is connected to again. The doc for this method indeed says

Attempts to connect to a peripheral don’t time out. To explicitly cancel a pending connection to a peripheral, call the cancelPeripheralConnection(_:) method

Which means that calling central.connect(peripheral) while peripheral is nowhere around to be seen, would stay pending, and iOS would trigger Bluetooth-related methods as soon as peripheral is seen again.

So, I did the following:

  • in centralManager(_:didDisconnectPeripheral:error:) and centralManager(_:didFailToConnect:error:) I request some spare time in background via UIApplication.shared.beginBackgroundTask in order to start a timer
  • the timer in its turn calls connect(_:options:) If this timer is set to 60 secs this would make 2 backgrounded apps exchange tempIDs for about 12 mins. However, after setting this timer to 15 secs I'm getting long time of exchanging tempIDs between 2 backgrounded apps. At least I had them doing so for 2 hours until I gave up myself and turned it all down. This is on fully charged phones, I suppose things are not that cool on discharged ones, haven't had time to test it yet.

This still doesn't fully solve the problem of inability to scan and find between 2 newly seen devices when both of apps are backgrounded. But it allows to keep tracking 2 people whose phones already found themselves in order to determine how long they were in contact with each other. Please, let me know if you are interested in a PR.

I would be interested in seeing a PR. I feel like at the moment Apple-Google APIs aren't going to be feasible to implement as it would require all of the users to be on the latest iOS version.

From what you said, does that mean the only case you are not covering at this point is when both devices are locked?

EDIT: I have read a few articles about how Australia was able to go around these issues and was wondering how they were able to do this? I think they mentioned how it was able to work as long as you locked your phone while the app was on the foreground. Any idea how they might've done this?

vitalii-tym commented 4 years ago

From what you said, does that mean the only case you are not covering at this point is when both devices are locked?

No, I meant that since the app acts both, as transmitter and scanner at the same time it is still capable to work if one of the two iPhones is locked (the foregrounded one is capable to scan a backgrounded transmitting one and they exchange tempIDs as the result).

To sum it up:

I would be interested in seeing a PR

There was a lack of interest and I saw the team wasn't reacting to other seemingly minor PRs, so I was reluctant to volunteer my time to wrap up my quick experiments into a working improvement. And it does require time, because the following things have to be done in order to make it fully working:

Since my last experiments the second iPhone I was using for it has travelled away with its owner and there is no other device I could use for debugging around me.

Here's my test code for reconnecting, which makes two backgrounded apps continue exchanging tempIDs for indefinite time:

var backgroundTaskId: UIBackgroundTaskIdentifier = .invalid
let reconnectInterval = 15 // seconds
private func tryReconnect(_ central: CBCentralManager, to peripheral: CBPeripheral) {
    DispatchQueue.main.async { // while in background mode Timer would work only being in main queue
        self.backgroundTaskId = UIApplication.shared.beginBackgroundTask (withName: "reconnectAgain") {
            UIApplication.shared.endBackgroundTask(self.backgroundTaskId)
            self.backgroundTaskId = .invalid
        }

        self.timerForScanning?.invalidate()
        self.timerForScanning = Timer.scheduledTimer(withTimeInterval: TimeInterval(reconnectInterval), repeats: false) { _ in
            central.connect(peripheral, options: [:])

            UIApplication.shared.endBackgroundTask(self.backgroundTaskId)
            self.backgroundTaskId = .invalid
        }
    }
}

func centralManager(_ central: CBCentralManager, didFailToConnect peripheral: CBPeripheral, error: Error?) {
    self.tryReconnect(central, to: peripheral)
}

func centralManager(_ central: CBCentralManager, didDisconnectPeripheral peripheral: CBPeripheral, error: Error?) {
    self.tryReconnect(central, to: peripheral)
}

Note that it would work only after one of the devices has found another (which needs one of them to be in foreground), also the reconnection would still work if the other device goes out of range and then becomes seen again.

Shailevy commented 4 years ago

Has anyone made any progress with this? Did anyone test if iOS 13.5 behaves the same?

gonharry commented 4 years ago

I got the same problem trying to force background to listen/transmit like a beacon on my project. This is what I found, but now I'm not able to test it, if some of you want to see it:

iBeacon when app is dead and gone

surya995 commented 4 years ago

Yesterday TraceTogether new IOS version released with the background (as long as you don’t force close the app) fix. with which option they fixed it.

subhransu commented 4 years ago

So it works on iOS 13.5 only? Just curious what changed as the limitations on background advertisement is still there.

All service UUIDs contained in the value of the CBAdvertisementDataServiceUUIDsKey advertisement key are placed in a special “overflow” area; they can be discovered only by an iOS device that is explicitly scanning for them.

If I am not wrong, the above limitation has not changed?

rostislawko commented 4 years ago

We can run a player using AVAudioSession with volume 0 (which can be a silent player). We will start this player once the app enters background and we will stop the player on didBecomeActive or enterforeground.

We can keep the app running in background by playing the silent audio periodically.

What if user opens Apple Music or different player? Will we lose background work until user opens app manually?

c19x commented 4 years ago

Dear OpenTrace, I've created a solution that supports constant background scanning and device proximity tracking on iOS and Android. The solution does not rely on keeping the iPhone unlock and using a black screen to reduce power usage. It acts like an ordinary app, where the user can background the app, use other apps, and lock the phone indefinitely, while it continues to detect and track other beacons. The contact tracing app will work on 81.56% of devices globally without software upgrade to latest version. It does not rely on the Google + Apple API as I fear many users won't or can't upgrade their OS, thus it will only work on a theoretical maximum of 76.98% of devices today. The Bluetooth beacon underpinning the solution will work on 98.74% of devices globally (June 2020). It can even handle Android devices that can only act as a central, not a peripheral (e.g. Samsung J6). Please take a look at https://github.com/c19x

The scan-connect-read-disconnect-pendingConnect|scan approach on iOS will fail when both devices are in background state (i.e. app not in foreground, device is locked) which will be the norm. They simply won't detect each other, until one of them comes to the foreground. Try the following ...

Failure condition 1

iPhone A and B in separate faraday bags (i.e. out of range of each other) Take A out, install and start app on A, move app to background, lock device, place back in faraday bag Take B out, install and start app on B, move app to background, lock device, place back in faraday bag Wait 30 minutes. This is equivalent to two isolated users installing their app and venturing out later Take A and B out, they won't detect each other until either A or B brings app to foreground Failure condition 2

  1. Continuing from 5, now that A and B are together and detecting each other.
  2. Move app on A and B to background, lock devices. Place A in faraday bag
  3. Wait 30 minutes. This is equivalent to two people meeting, then going out of range of each other for a while.
  4. Take A out of faraday bag, they won't detect each other until either A or B brings app to foreground

This, and other related issues, have been investigated and a solution has been developed and documented for sharing. Please take a look at https://github.com/c19x/C19X-iOS-BLE Best regards and good luck 👍

sfinktah commented 4 years ago

Yesterday TraceTogether new IOS version released with the background (as long as you don’t force close the app) fix. with which option they fixed it.

Whilst in Australia, our App is still on Version 1.x, Singapore has released 2.1 -- reportedly solving iOS based issues.

However, there have been no notable updates to this repo to see how the issue was solved -- if it was -- or what limitations there are.

abhinaynatraj commented 4 years ago

@vitalii-tym Ios 13.5.1 hasn't fixed any of these issues for you correct? I'm still having the same issues with background and stuck

apecodingman commented 4 years ago

iOS restriction explanation

Finding A:

https://medium.com/@cbartel/ios-scan-and-connect-to-a-ble-peripheral-in-the-background-731f960d520d Active vs Passive scanning This information helped us locate the following Apple Developer post which describes some important undocumented behavior of iOS devices; iOS devices perform Active scanning while in the foreground and Passive scanning while in the background. This page provides a good description of the difference between Active and Passive scanning: When not connected, Bluetooth devices can either advertise their presence by transmitting advertisement packets or scan for nearby devices that are advertising. This process of scanning for devices is called device discovery. There are two types of scanning; active and passive. The difference is that an active scanner can send a scan request to request additional information from the advertiser, while a passive scanner can only receive data from advertising device. Note that the terms discovery and scanning may be used interchangeably. The figure below shows the sequence where the scanner sends a scan request to an advertiser during an advertising event. In other words, while the iOS device is in the foreground and Actively scanning, it will receive Advertising packets, send out Scan Requests, and receive Scan Response packets. In the background, the iOS device will only receive Advertising packets.

Finding B: https://developer.apple.com/library/archive/documentation/NetworkingInternetWeb/Conceptual/CoreBluetooth_concepts/CoreBluetoothBackgroundProcessingForIOSApps/PerformingTasksWhileYourAppIsInTheBackground.html

All service UUIDs contained in the value of the CBAdvertisementDataServiceUUIDsKey advertisement key are placed in a special “overflow” area; they can be discovered only by an iOS device that is explicitly scanning for them.

Finding C: see the method description of below function, when scan in the background, service UUID(CBAdvertisementDataServiceUUIDsKey) must be specified and can not be nil. And this service UUID must be matched during the pairing. open func scanForPeripherals(withServices serviceUUIDs: [CBUUID]?, options: [String : Any]? = nil)

Finding D: When the app broadcasts the bluetooth signal in the foreground, both CBAdvertisementDataServiceUUIDsKey and CBAdvertisementDataOverflowServiceUUIDsKey can be printed in the advertisementData. However, when in the background, only CBAdvertisementDataOverflowServiceUUIDsKey can be printed in the advertisementData.

Finding E: When two iOS devices both run the bluetooth scan&broadcast in the background:

  1. The device is in passive scanning and can only discover the bluetooth signal with CBAdvertisementDataServiceUUID in the advertisementData

  2. The device is broadcasting only with CBAdvertisementDataOverflowServiceUUID in the advertisementData, no !!!!!!!!!!!CBAdvertisementDataServiceUUID

    With above findings, we can see when the iOS device is broadcasting in the background, only CBAdvertisementDataOverflowServiceUUID is included in the advertisementData. Thus, only active scanning or explicitly scanning as Apple mentioned can find this background broadcast. So there's no hope that Apple will fix this and this is the design of Apple Bluetooth and not a defect.

sfinktah commented 3 years ago

Did this eventually get fixed, we're at iOS 14.6 now.

lokeshiih commented 2 years ago

Consider on above one, we can say that how iOS broadcasting in the background. I appreciate your efforts, Thanks a lot for sharing your thoughts. I think its beneficial for iBeacon App Development

victorstewart commented 2 years ago

hi @c19x

when you said...

"Herald works around this by using Bluetooth callbacks to trigger an activity loop, effectively waking Herald’s activity if another device running Herald is nearby. Herald also ‘pushes ahead’ a timer by a few seconds, within the ten second limit of iOS background app activity, so that the timers stay effective over a long period of time."

were you implying that by stopping and restarting a scan that the central will "rediscover" peripherals that it has already seen, and be woken up by these discoveries, whereas otherwise they would be ignored as "already seen recently"?

and if i may, do you have any idea when these "already seen recently" limits expire so that peripherals would be rediscovered by a persistent scan? or by what other factors peripherals will be rediscovered?

thanks!

littlecircle0730 commented 1 year ago

Is this solution still working? The repository has been removed. Where could we find it? Thank you.

sfinktah commented 1 year ago

BlueTooth is dead. Long live iBeacons, which suffer inferior support on Android :p

WebelightSolution commented 4 months ago

Great article! Overcoming the iOS background Bluetooth limitations can indeed be quite challenging, especially with the restrictions Apple imposes for security and battery optimization. Your insights on optimizing the timing and leveraging BLE peripheral mode are very helpful.

For anyone looking to implement these solutions effectively, I highly recommend checking out Webelight Solutions. We have a strong track record in developing innovative iOS applications that work seamlessly with Bluetooth technologies.

You can learn more about their services at Webelight [Solutions]

Keep up the good work and thanks for sharing these valuable tips!