Databackup Service

[pchainho]

As discussed here the idea is to have a generic Databackup Service to backup user's core data and also sync data between different runtimes even if the runtimes are not simultaneously connected eg policies, contacts, dataobjects, etc

The proposal is to use the data sync framework with a databackup objects. The Storage Manager would be extended (the set() function) to be the reporter of Databackup objects and the Databackup Observer hyperty would run in the Nodejs Runtime updating the storage manager every time there is a change performed by the reporter. All users' runtime would also be observers of these databackup objects in order to ensure complete sync among them. When a new runtime is instantiated it will resume the Dataobjects sync with the most updated versions from the Databackup observers.

The User should be able to set the Databackup service provider by using the Runtime Admin GUI, similar to the IdP. In this way the user also has the power to decide whom to trust his own data.

It should also be noted that existing (legacy) Cloud based backup services (like dropbox, google drive, etc) can also be integrated by using the legacy interworking framework ie only appropriate IWProtostub and IWIdpProxy have to be provided with no impact on other components.

This is a quite powerful feature that would require a minimum development effort.

Additional details:

• extend storageManager.set(..) with "backupAuthorised : userURL[]" input parameter defining the users that have permissions to read the Backup data object from the databackup service.
  • backupAuthorised is added to the invitation request body sent to Databackup observer.
  • the Databackup Observer adds a listener to the address in order to handle Databackup read requests.
  • Read requests are allowed for backupAuthorised
  • new "public" attribute to SyncDataObject that if true allows anyone to read the Databackup object.

[pchainho]

To clarify these questions:

• the StorageManager constructor would have the RuntimeURL and msg bus as input parameters

• the Storage Manager would have the address: hyperty-runtime://<runtime-instance-identifier>/storage

• the Syncher owner is the Storage Manager Address

[pchainho]

Evaluate the implementation of this feature with dexie.syncable.

Initial notes:

1. add optional boolean input field backup to syncher.create(..) to enable backup feature for the created Data Object.
2. StorageManager constructor with remoteStorage option to create a DB synched with a remote storage. Use Dexie related sync mechanisms at Storage Manager implementation of toolkit and browser runtime
3. runtime/Storage with new exported function createSyncDB
4. DataObjectStorage: one storage per remotely synched DO:
   1. _storeDataObject with list of DOs urls remotely synched
   2. _storeDataObject would also contain in memory DOs remotely synched
   3. with list of remotely synched storages
   4. loadRemote() new function to load and initialise remotely synched DOs. To be called by RuntimeUA
5. SyncManager would create a separated backup enabled DB for each DO to be synched with remote server using DO URL as db name (tbc).
6. on DO creation request, sync manager would use SyncSyncManager when backup is enabled.
7. the URL of the remote SYnc Server would be a new field in the DO metadata.
8. on request to sync (resume) with reporters if there is a timeout, the observer would connect with Reporter Sync Server to sync with it. As soon as data is retrieved it disconnects from the sync server.
9. for Data Objects with child objects that are backup enabled, when the Reporter is offline, a connection to the Sync Server of the Reporter is established.