Merges, revisited - Githubissues

mediachain / concat

Mediachain daemons

MIT License

42 stars 13 forks source link

Merges, revisited #68

Closed vyzo closed 7 years ago

vyzo commented 7 years ago

Improvements related to merges:

Add deps field to the SimpleStatement protobuf message for object dependencies [#63]
Fetch object dependencies during merge
Optimize merge performance with data fetch parallelization and statement merge batching [#44]
Add a /data/merge/{peerId} endpoint to support raw data merging (outside query merges)

Closes #44 Closes #63

vyzo commented 7 years ago

Final tweak: delayed the datastore occurs check until request time, which minimizes the window for duplication and also concurs a tiny bit of perf improvement in large merges due to the parallelization of the occurs check.