experiment with data sync - Githubissues

etenlab / docs

Documentation over all projects in the ETEN Innovation Lab repos

MIT License

1 stars 0 forks source link

experiment with data sync #300

Open michaelmarshall opened 1 year ago

michaelmarshall commented 1 year ago

I want to experiment with a different way to do the data sync.
the tables we sync will be the 6 graph tables and the 4 crowdsourcing tables (nodes, rels, node/rel prop-keys, node/rel prop-values, elections, candidates, votes, users)
1. put the tables in a dto type:
```
export type DatabaseDTO = {
nodes: Array<Nodes>,
node_property_keys: Array<NodePropertyKey>,
node_property_values: Array<NodePropertyValue>,
relationships: Array<Relationship>,
relationship_property_keys: Array<RelationshipPropertyKey>,
relationship_property_values: Array<RelationshipPropertyValue>,
users: Array<User>,
elections: Array<Election>,
candidates: Array<Candidate>,
votes: Array<Vote>,
}
```
2. Put the database into the DTO then gzip the json as a file: db.json
3. The data will be transferred via these files. Which should make offline transfers easier.
4. When a client initializes it sends the gzip'd json of its database to the server. The server parses it and combines it with the server database.
5. The server then sends a gzip'd file of the new combined database back, with any duplicate data removed that the client already has.
6. The client merges the new file with its database.