[FEATURE] Real-time Account Data Sync

[msprijatelj]

Is your feature request related to a problem? Please describe. I am frustrated with the current save/load synchronization functionality in CompCon. Users frequently encounter data loss due to sync issues, and often receive conflicting advice on the PilotNet server on best ways to avoid data loss (e.g. turn on Auto Save vs turn off Auto Save). The lack of instantaneous data synchronization between instances of CompCon also leads to data loss issues and limits reliable app usage to a single device and browser tab.

Describe the solution you'd like Implement a real-time data storage and synchronization solution for CompCon. Since CompCon uses an AWS backend, leveraging AWS AppSync may meet the purposes for this feature. Ideally, use AppSync to:

• Synchronize account information between the browser, the cloud backend, and any other login instances in real time
• Synchronize pilot data between all users that possess the pilot’s vault code
• Establish an offline version of the data store when logged out or disconnected from the internet
• Provide resolution between conflicting data sources

Describe alternatives you've considered This would require a larger overhaul and porting data out of AWS, but using a Firebase Realtime Database EDIT: Google Cloud Firestore could provide similar functionality to AppSync, while tying together security through Firebase auth services and potentially obviating the need for an S3-type service. Depending on CompCon’s usage metrics, it may even be more cost-effective than AWS.

Additional context I am of the opinion that shifting away from an (auto)save-based to real-time data handling will drastically improve the app experience for the end user. With users familiar with web apps like Google Sheets and D&D Beyond that sync their information for all users in real time, using services like AWS AppSync or Firebase Realtime Database will enable intuitive usage of CompCon.

Contact Info Valkyrion#6196

[jarena3]

I've been thinking about this quite a bit.

My principal concerns are the cost (though the recent changes may help with that) and managing the startup/data load process, which would (I think) necessitate a pretty thorough rework of object hydration and saves. It'd require some heavy lifting, but I think it would be possible if cloud accounts were mandatory, rather than optional. I know AppSync's storage engine can handle userless offline, but I don't know how carefully it's managed.

I think maybe in the short term it'd be best to drop auto-sync and better integrate manual sync backed/supported by an appsync update timestamp (per saved item). This should at least get me to a point where users have visibility into and control of what data is getting saved, and where.

It also may be worthwhile to split off active mode (at least in the backend) and manage that separately, where the data relevant to a single active instance could be more carefully managed.

[msprijatelj]

@jarena3 Thoughts on this issue now that the Cloud Rework has released? I imagine that handling cloud vs local version control has covered most of this issue. AWS AppSync may still be useful for future Active Mode/Mission runner "rooms"? Dunno if this issue should stay open as reference for future Active Mode work, or close since the Cloud Rework is live.

[msprijatelj]

Dropping another comment here to mention that Google’s Firestore is more recent and fitting for our needs than their Realtime Database: https://firebase.google.com/docs/firestore/manage-data/enable-offline

To use offline persistence, you don't need to make any changes to the code that you use to access Cloud Firestore data. With offline persistence enabled, the Cloud Firestore client library automatically manages online and offline data access and synchronizes local data when the device is back online