atsign-foundation / at_server

The software implementation of Atsign's core technology
https://docs.atsign.com
BSD 3-Clause "New" or "Revised" License
40 stars 12 forks source link

ability for sync verb handler to skip delete commit entries to be sent to client #2097

Open murali-shris opened 2 months ago

murali-shris commented 2 months ago

Is your feature request related to a problem? Please describe.

https://github.com/atsign-foundation/at_server/issues/2066 "Excessive Syncing of Deleted Keys: New clients must sync all keys, including numerous deletions, leading to significant time and space inefficiencies."

Describe the solution you'd like

Describe alternatives you've considered

No response

Additional context

No response

murali-shris commented 1 month ago

Complete some of the changes on server to skip delete entries. While writing tests encountered a bug in _acceptKey in commit log keystore. https://github.com/atsign-foundation/at_server/pull/2115 Will fix the bug and then resume work on this feature

murali-shris commented 1 month ago

Server changes: https://github.com/atsign-foundation/at_server/issues/2097 Client changes: https://github.com/atsign-foundation/at_client_sdk/tree/sync_skip_deletes End2End testing in progress.Once successful, will run performance tests. Server PR to be merged first once performance results are good

murali-shris commented 2 weeks ago

Implemented skipUntil changes and extensive unit tests on server https://github.com/atsign-foundation/at_server/pull/2131 Changes in at_client in progress

murali-shris commented 2 days ago

https://github.com/atsign-foundation/at_client_sdk/pull/1432 At client changes complete.Will re-open PR once benchmarking tests are complete and server changes are merged Performance bench marking in progress https://docs.google.com/spreadsheets/d/1Mc4GFe2-xZ4CZImBa3xWiAufq7PaOm42c3pItTYSPXQ/edit?gid=0#gid=0