Zombies vs Orphans

[githubbob42]

Mingle Card: 4319 Analysis

For a given object the md5 checksum of the joined ids could be compared with the server to determine if anything has changed.  Anything which would normally be picked up by the regular incremental last modified check should be ignored.  If the 2 checksums do not match, then reset the last sync date for that data type and clear the store.   Below are 2 pieces of code for generating the checksum.

We want to only apply this change to transaction data, and only if sharing rules apply. Otherwise we want to incrementally sync as normal.  I would suggest statically controlling which types do this for now and add some dependency tracking later to support fxforms later.

20160920 – DP: this would also cover cases where computed AltSync flags would change value.

Steps:

1. Red Sync all of the transaction data. (This is determined by the existence of an externalId which is either SyncId or TrackingNumber)
2. Add the above class to the server code.
3. Add the js-md5 module to alpine-mobile
4. In the sync/index module an event listener needs to be added for describe
   • we add the description result to the request
   • if it’s transactional and there is a lastSyncDate, we register a redSyncCheck for that object type
   • If it’s not transactional, we call directly into incremental
5. The redSyncCheck listener would then branch, either calling the redSync method, or the incremental method to query the data.

Questions:

I wonder if there would be a way to grab the checksum for a whole group of items.   Perhaps the whitelist could be used to cut down on the number of requests.

Alternatives

1. Split the orphan and addition problems.  The additions would be handled by joining with the share table and comparing the last modified of the share rows and the last modified of the data rows to the last sync date.   The orphans would be handled post-sync as a background process.  Potential pitfalls:
   1. The join logic for the last modified on the share table would increase the potential for paging and add quite a bit of complexity to the queries.
   2. Iterating over a table to remove orphans while the user uses the app in the foreground could cause performance issues.  Web apps are single threaded and some processing on indexeddb may block the UI.  This could lead to long pauses or jitters.
   3. The user experience might be strange.  As the user is looking at a list of jobs, jobs may get removed by the background process.
   4. Finally, for large tables (this was proposed for lookups as well)  we would still be pulling down a substantial quantity of data.

Related Stories

4318 is a pre-requisite for this story.

Tasks

{{table query: SELECT Number, Name, Owner, 'Task Status' WHERE Type = Task and Story = THIS CARD}}

Defects

{{ table query: SELECT Number, Name, Owner, 'Status' WHERE Type = Defect and 'Related Story' = THIS CARD }}

Test Plan

[githubbob42]

Feature: #3972 Sharing Rules for Incremental Sync (Mingle)