mozilla-services / syncserver

Run-Your-Own Firefox Sync Server
Mozilla Public License 2.0
1.86k stars 142 forks source link

What data will be synced? Addon Data? #213

Open ghost opened 4 years ago

ghost commented 4 years ago

Hey Team,

sorry i ask a question here but i have no idea where else i could ask.

I wanted to know what data exactly will be synced to the selfhosted Firefox-Syncserver? As i saw the default data from my profile will be synced (Bookmarks, History, Passwords,...) How does it look like about addon data that syncs into the cloud?

Thanks ;)

jrconlin commented 4 years ago

Hi, First off, any sync data stored is encrypted by the browser*. The encryption keys are derived from some mozilla info as well as the passphrase you use when setting up sync. This is one of the reasons that you can lose data if you forget your password.

As for what data is synced, it kind of depends. In firefox you can specify what data you want synced across your devices in the about:preferences#sync tab. That's not per browser instance (for example, your mobile phone being different than your laptop), but across all instances of Firefox.

Self hosting a server means running pretty much the same thing we do on our servers. We just store the actual data a bit differently and have to worry a bit more about getting things organized at a larger level. We're working on a new sync server, but stand-alone will be the same code base we use in production.

* There is some "meta data" information that isn't encrypted. The meta data tells us things like when you last sync'd and what types of info you've sync'd but any actual data like passwords or history is all encrypted and indecipherable to us or anyone else.

ghost commented 4 years ago

Thanks for clarification and i understand it so far, but what i still dont know what happens with AddonData that allow CloudSync, because this data is not specified in the about:preferences#sync Tab. Addons that support cloud sync often has an button to start a backup and a button to restore. Is this data also be saved on the own cloudserver?

jrconlin commented 4 years ago

Yeah, the sync control panel only really mentions a set of data that's synced. The longer version is that each user has a set of "collections". Each collection is either "well-known" or is a "user-collection". This is a bit of legacy from very early on where sync would host all sorts of data, but the general work would be exactly the same no matter what the "collection" is.

I should also take a moment to say that the back end server isn't very smart, by design. How data is packed up for synchronization or unpacked for rectification is done by specific code in the browser. It might be good to think of the server as a delivery service. They don't care what's in the box, just that it gets to where you want it to go.

In the case of "add-ons", it's just another collection type. Add-on information is boxed up, and handed to sync so that it's copied to the other browser instances.

One note of caution about your question: syncserver isn't really a backup system. Older sync really wasn't either. We're fixing that with the new server. It works as a backup much like sending money to yourself makes the postal system a bank. It works a lot of the time, and usually does what you think, but if anything goes wrong, you're going to be sadder than you expect. Fortunately, for stand alone instances, it's a bit better since you have more control over the database and the machines you're running on.

rfk commented 4 years ago

How does it look like about addon data that syncs into the cloud?

It depends very much on the addon in question.

For addons that use the chrome.storage.sync API to store data and have it sync between your different devices, then we're doing some work in the latest version of Firefox which will cause that data to be stored on the self-hosted server alongside your history, bookmarks, etc.

For addons that implement their own storage and syncing APIs, they will not use the self-hosted sync server (and they would typically be storing their data in a service that's not run by Mozilla at all).

Do you have a specific add-on in mind that you're curious about here?

ghost commented 4 years ago

How does it look like about addon data that syncs into the cloud?

It depends very much on the addon in question.

Do you have a specific add-on in mind that you're curious about here?

For example the Multi Account Containers Plugin. As i see this Addon uses the Firefox Cloud storage. I had many problems the last weeks about the bug with temporary containers so i thought about it. https://github.com/mozilla/multi-account-containers/issues/1675

rfk commented 4 years ago

Yes, that one does use the chrome.storage.sync API. In current release Firefox it will not sync data to a self-hosted server, even if all your other data is syncing there. In the current development version of Firefox we've recently landed code that will make it sync to the same server as your other browser data, including a self-hosted server if configured.

ghost commented 4 years ago

Cool thats cool news ;)

Thanks for the information

jackyzy823 commented 4 years ago

@ryperx This wiki shows what syncserver synced : https://mozilla-services.readthedocs.io/en/latest/sync/objectformats.html

chrome.storage.sync which used by extensions is decided by services.sync.prefs.sync.webextensions.storage.sync.kinto webextensions.storage.sync.enabled webextensions.storage.sync.kinto webextensions.storage.sync.serverURL in about:config. currently the server is https://webextensions.settings.services.mozilla.com/v1 . however you could self host this too with kinto and kinto-fxa.

and kinto is https://github.com/Kinto/kinto.

maybe they want to replace kinto with syncserver finally? :) see https://bugzilla.mozilla.org/show_bug.cgi?id=1635352 and https://searchfox.org/mozilla-central/source/services/sync/modules/service.js#110

mhammond commented 4 years ago

maybe they want to replace kinto with syncserver finally? :)

Yes, since Firefox 79, browser.storage.sync addon data is handled by the normal sync infrastructure instead of kinto.