mozilla-services / syncstorage-rs

Sync Storage server in Rust

Mozilla Public License 2.0

896 stars 49 forks source link

Document our Spanner architecture #589

Open pjenvey opened 4 years ago

pjenvey commented 4 years ago

We've learned a lot about Spanner while building Durable Sync, let's extract our learnings into a high level doc describing our Spanner schema, decisions behind it, possible improvements, observations on its real world usage, etc. This could just be a .txt file living inside the repo.

Possibly a separate issue: we should document what Spanner metrics to monitor under load testing. This might be appropriate residing in our Runbook.

Emmanuel-Melon commented 4 years ago

@pjenvey I'd like to work on this because I have been meaning to get into Spanner and documenting it would be a great learning opportunity.

fzzzy commented 4 years ago

@Emmanuel-Melon There is now an implementation of spanner that you can run locally. We haven't tried it yet, but if you would like to try to get it working and document it that would be helpful.

https://github.com/GoogleCloudPlatform/cloud-spanner-emulator

Emmanuel-Melon commented 4 years ago

@Emmanuel-Melon There is now an implementation of spanner that you can run locally. We haven't tried it yet, but if you would like to try to get it working and document it that would be helpful.

https://github.com/GoogleCloudPlatform/cloud-spanner-emulator Okay great, I'll run this then update you!

Emmanuel-Melon commented 4 years ago

@Emmanuel-Melon There is now an implementation of spanner that you can run locally. We haven't tried it yet, but if you would like to try to get it working and document it that would be helpful.

https://github.com/GoogleCloudPlatform/cloud-spanner-emulator

I just got it running locally.

pjenvey commented 3 years ago

877 added some documentation for batch commit (describing its count of mutations)