Mingle Card: 3068
The implementation can be straightforward based on which option we select, but whatever solution needs to be scalable as we add more objects.
Some of the strategies incldue:
1.) Have Client sync iterate through all downloaded objects at the end and count all records. Some of what need to solve around the download package:
The common pattern around objects and their data sent down to the client is documents."objectName".data = array of items.
some objects are mapped in a different form
Ex:
eforms has an associate array that we need to documents.eforms."object with each object being the key, and their data(array of items) being the value.
userprofile comes down as an object(only ever 1 row)
attachments and pricebookItems(which are not part of the sync package and come down seperately).
2.) Have each module in the client(that handles downloaded data publish out their row count.
Will mean modifying the onSyncDataReady handler in each module to publish out their row count on sync. (or, a much efficient solution, have each storage module expose/manage a row count property)
All future modules that provide onSyncDataReady will need to support a row count, which isn't so bad as a convention(Should just cache that info
3.) Resolve row count on the server sync steps and pass it down to the client.
Currently fetch lookup data, transactional data, pricebook data, and attachments are seperate requests to the server. Each sync step will calculate and store a row count property on the root level(Ex: data, metadata, count). Any future sync step's responsibility would be to include a count property on the object.
Client would simply read over each count property documents."objectName" and pass back a total to the server. (Seems odd that we would get counts from the server, then send it right back to the server for metrics. Maybe a better solution is all sync requests write out their stats to mongodb and the server, and the server(or a seperate job) will scan through, combine the records and writes it out to the log
4.) Resolve row count on server and publish out metrics using correlation id
Currently fetch lookup data, transactional data, pricebook data, and attachments are seperate requests to the server. Client will provide a correlation id to all lookup querys, transactional query, pricebook queries, and each sync component will publish out their own stats on sf records for librato metrics. Each component will tag each count with the correlation id to allow us to properly report on row counts per sync.
githubbob42
commented
5 years ago
Mingle Card: 3068 The implementation can be straightforward based on which option we select, but whatever solution needs to be scalable as we add more objects.
Some of the strategies incldue:
1.) Have Client sync iterate through all downloaded objects at the end and count all records. Some of what need to solve around the download package:
The common pattern around objects and their data sent down to the client is documents."objectName".data = array of items.
some objects are mapped in a different form
Ex:
2.) Have each module in the client(that handles downloaded data publish out their row count.
3.) Resolve row count on the server sync steps and pass it down to the client.
4.) Resolve row count on server and publish out metrics using correlation id