Calling dispatch in applicationDidEnterBackground sends events multiple times

[makadev]

If dispatch is being used as documented under Manual dispatching, it might send the same events again.

As mentioned in #311, the connection is being closed with a TimeOut Error when the App enters Background. But in difference to a real Connection or Send TimeOut the request will be send and handled by the remote server.

I could reproduce the Problem with MatomoTracker 7.2.0 on iOS 13.2.2 and iOS 12.4.x as follows:

1. Trigger one or more event's which do not hit the Time or Event count limit for dispatching and put the App in Background.

2. When applicationDidEnterBackground call's dispatch all queued events will be send but the connection fails:

   * MatomoTracker [I] Start dispatching events
   * 2019-12-04 12:59:41.315754+0100 App [x:x] Can't end BackgroundTask: no background task exists with identifier 7 (0x7), or it may have already been ended. Break in UIApplicationEndBackgroundTaskError() to debug.
   * 2019-12-04 12:59:57.649216+0100 App [x:x] Task <81E043A6-BD27-4E81-8FEA-FC86DDFFD742>.<200> finished with error [-1001] Error Domain=NSURLErrorDomain 
   Code=-1001 "Zeitüberschreitung bei der Anforderung."
   UserInfo={
   NSUnderlyingError=0x60000340d380 
   {
   Error Domain=kCFErrorDomainCFNetwork 
   Code=-1001 "(null)"
   UserInfo=
   {
     _kCFStreamErrorCodeKey=-2102,
     _kCFStreamErrorDomainKey=4
   }
   },
   NSErrorFailingURLStringKey=https://ww.xx.yy.zz/piwik.php,
   NSErrorFailingURLKey=https://ww.xx.yy.zz/piwik.php,
   _kCFStreamErrorDomainKey=4,
   _kCFStreamErrorCodeKey=-2102,
   NSLocalizedDescription=Zeitüberschreitung bei der Anforderung.
   }
   * MatomoTracker [V] MatomoTracker is already dispatching.
   * MatomoTracker [W] Failed dispatching events with error 
   Error Domain=NSURLErrorDomain 
   Code=-1001 "Zeitüberschreitung bei der Anforderung."
   UserInfo={
   NSUnderlyingError=0x60000340d380 
   {
   Error Domain=kCFErrorDomainCFNetwork 
   Code=-1001 "(null)"
   UserInfo=
   {
     _kCFStreamErrorCodeKey=-2102,
     _kCFStreamErrorDomainKey=4
   }
   },
   NSErrorFailingURLStringKey=https://ww.xx.yy.zz/piwik.php,
   NSErrorFailingURLKey=https://ww.xx.yy.zz/piwik.php,
   _kCFStreamErrorDomainKey=4,
   _kCFStreamErrorCodeKey=-2102,
   NSLocalizedDescription=Zeitüberschreitung bei der Anforderung.
   }

   The localized error "Zeitüberschreitung bei der Anforderung." means that the request timed out.

3. Next time the App is opened, dispatch will send everything again:

   MatomoTracker [I] Start dispatching events
   MatomoTracker [I] Dispatched batch of 3 events.
   MatomoTracker [I] Finished dispatching events

4. Check Server logs (and Tracker), it shows both requests from step 2./3. where handled:

   ww.xx.yy.zz - - [04/Dec/2019:11:59:41 +0000] "POST /piwik.php HTTP/1.1" 200 358 "-" "Mozilla/5.0 (iPad; CPU OS 13_2_2 like Mac OS X) AppleWebKit/605.1.15 (KHTML, like Gecko) Mobile/15E148 MatomoTracker SDK URLSessionDispatcher"
   ww.xx.yy.zz - - [04/Dec/2019:12:00:27 +0000] "POST /piwik.php HTTP/1.1" 200 358 "-" "Mozilla/5.0 (iPad; CPU OS 13_2_2 like Mac OS X) AppleWebKit/605.1.15 (KHTML, like Gecko) Mobile/15E148 MatomoTracker SDK URLSessionDispatcher"

--

Only mitigation I see is to not use manual dispatching at points, where the app enters background f.e. in applicationDidEnterBackground or when opening external apps. Note that the Problem still exists without manual dispatching, not very likely to be triggered but it's possible.

[brototyp]

Hi @makadev, thanks for the report.

I am currently checking with the backend team, if there is a way to reliably mitigate this by e.g. sending some kind of identifier along with a request/event that can be used to unique them on server side. But I am not sure if that's possible yet, or will be possible in the near term.

In the meantime I think we should either update the documentation, or fix #311. What do you think?

[makadev]

Hi @brototyp, thanks for the answer.

In the meantime I think we should either update the documentation, or fix #311. What do you think?

Maybe both.

I'm cautious about the proposed solution from #311 since I never used Background Sessions but from what I read it might work by creating a custom Dispatcher like an URLSessionDispatcher variant using Background Sessions and let the user decide, which dispatcher is to be used.

[brototyp]

That sounds like a great idea.

Short term solution/mitigation

Let's update the documentation asap as a short term "solution"

Long term solution

• Create a dispatcher that uses a background session to dispatch events

• TBD: Is there ever a reason not to use the background-session dispatcher? Should we let the user of the SDK decide which dispatcher to use, or just provide a single one?

• Another approach could be to create a background task before sending the request and finishing it after the response is processed. This is the way the Google Analytics SDK did solve it back in the days.

[brototyp]

Hi @makadev, I just created this PR. My approach there is to wrap every sending of the events into a background task. Can you try it out and give me some feedback?

[makadev]

Hi @brototyp,

I've tested it with the dispatch in applicationDidEnterBackground and the BackgroundDispatcher is working as expected. The POST is handled in background and there are no more bogus connection issues and duplicates. 👍