Make Sync Server Url configurable in the UI

[iWangJiaxiang]

Staff edit (by @bsclifton)

Custom sync URLs are supported. You can use the command line flag: --sync-url=https://sync-v2.brave.com/v2

And replace https://sync-v2.brave.com/v2 with the server of your choosing. Our server implementation is open and can be found here: https://github.com/brave/go-sync

The ask for this issue is to offer a place in the UI to edit the URL.

Description

To protect user data, it is important to support self-hosted sync service for advanced/professional users, for example, those users using NAS/HomeLab/HomeServer. Now the self-hosted feature of Brave sync server v2 is supported according to Issue #51 · brave/go-sync. The next step is supporting custom sync server url in brave-browser! Then the brave browser solution could be important part of homelab stack list!

[bsclifton]

@iWangJiaxiang this should already be possible - you can edit the shortcut (or however you launch Brave) to provide a different URL. The argument looks like this: --sync-url=https://sync-v2.brave.com/v2

[iWangJiaxiang]

@bsclifton Thanks for your quick reply!

Do mobile clients (Android/IOS) support this way?

[bsclifton]

@iWangJiaxiang good question - they support this, but there is no way to set this. I'll remove Desktop label from this issue and we can use it to track being able to change this server somehow

[llucax]

Is the brave sync server open source too? If so, where can I find it?

If Brave is still close enough to Chrome, it might be possible to still set the sync server for mobile (although using a very user-unfriendly way, I hope Brave doesn't follow this pointless policy of not allowing users to easily set up a custom sync server and eventually implements it in a more accessible way, even if it is still hidden in some advance configuration option).

[bsclifton]

@llucax the code for the server is open source and can be found here: https://github.com/brave/go-sync

[captainepoch]

this should already be possible - you can edit the shortcut (or however you launch Brave) to provide a different URL

Could it be possible to add an option into the Privacy & Security section, or when you're going to setup the sync for the first time? Firefox allows you to change the sync server URL in the about:config, which is easy enough to setup your own sync server.

I kow the about:config doesn't exist in Brave, but something easier than changing in the params of the launcher would be great!

[fastman]

yup, It'd be great to have this on both mobile and desktop!

[claudio4]

this should already be possible - you can edit the shortcut (or however you launch Brave) to provide a different URL

Could it be possible to add an option into the Privacy & Security section, or when you're going to setup the sync for the first time? Firefox allows you to change the sync server URL in the about:config, which is easy enough to setup your own sync server.

I kow the about:config doesn't exist in Brave, but something easier than changing in the params of the launcher would be great!

Actually, Brave has brave://flags/ (and a few more, full list in brave://chrome-urls/) , it might may be possible to add it there.

[FlavioSantoro92]

News about that? On mobile would be great to edit, I don't want to fork the repo and build each time only to change the sync url

[Joshndroid]

Still nothing as yet? Would love to see this supported in the android app.

[DaCHack]

@iWangJiaxiang this should already be possible - you can edit the shortcut (or however you launch Brave) to provide a different URL. The argument looks like this: --sync-url=https://sync-v2.brave.com/v2

Hi,

I started the Sync server via docker on my local system as described in the README.md, but when I start brave with brave-browser --sync-url=http://127.0.0.1:8295, brave://sync-internals/ gives me a 404 HTTP error and with https it gives me an ERR_SSL_PROTOCOL_ERROR.

[27504:27516:0807/232750.493541:ERROR:ssl_client_socket_impl.cc(981)] handshake failed; returned -1, SSL error code 1, net_error -107

Any idea what I am doing wrong?

Many thanks!

[bsclifton]

cc: @yrliou

[yrliou]

@DaCapitalist Could you try --sync-url=“http://localhost:8295/v2”, and open brave://sync-internals/ to verify Server URL is set to the value as expected?

[DaCHack]

@DaCapitalist Could you try --sync-url=“http://localhost:8295/v2”, and open brave://sync-internals/ to verify Server URL is set to the value as expected?

@yrliou Many thanks. It works with --sync-url=http://localhost:8295/v2 (without the quotation marks!) But is the data exchange now HTTPS encrypted or not? I thought this should be encrypted?

Thanks!

[yrliou]

@DaCapitalist Glad it works now. Your HTTP request & response itself between your browser and your local server is not encrypted, but the actual sync data that being passed in the request and response is always encrypted and will be decrypted on the client side only. (p.s.: meta data is not encrypted but that shouldn't be an issue.)

[DaCHack]

Okay, many thanks. This is just awesome! Only missing piece now is to set this URL also on mobile apps (especially iOS). Waiting for this!

[DaCHack]

@iWangJiaxiang good question - they support this, but there is no way to set this. I'll remove Desktop label from this issue and we can use it to track being able to change this server somehow

Should‘t this be a Basic UI change in all clients and not only mobile/android? Thus, I would vote for keeping this also in the desktop (and iOS) backlog. Funktionality should be already existing in Chromium so we only need a setting in the UI to change the Variable. E.g. As @claudio4 suggested. Plus, since firefox is already offering this setting I suggest to increase the priority. What do you think?

[DaCHack]

4 months after our discussion I wanted to bring this up again. How are we doing with this UI feature for desktop and mobile?

[JulienSambre]

Hi!

Same question here : this is the only thing that I need to migrate my family browsers to Brave (from Firefox + self-hosted sync server)

[ln-12]

Hey @bsclifton, is there any update on this? For me, it would be enough to set a URL once (e.g. via brave://flags/) on both mobile and desktop. Having it in the regular UI would be awesome, but having no option at all is a great pity. Is there a possibility yet?

[metal450]

Like others above, the only thing that keeps me on Firefox is Brave's lack of a proper option. As mentioned in https://github.com/brave/brave-browser/issues/20431#issuecomment-1013290190, using a command-line arg isn't really "safe", as if it's ever missed/forgotten, the browser will just silently publish your data on the public server. Is making this available in brave://flags/ really a large undertaking...?

[JamesJosephFinn]

Why has this basic feature request languished for over two years? Self-hosting a bravesync server is only practical/useful if we can set the self-hosted bravesync sever location within brave browser settings! Thank you to the amazing team at Brave, but this right here is a curious omission. Please advise.

[ppar]

you can edit the shortcut (or however you launch Brave) to provide a different URL

While this is technically true, it's not exactly a feasible solution.

On Linux (Debian), that change needs to go to the wrapper script /opt/brave.com/brave/brave-browser, but this file gets overwritten every time brave itself is updated - which is often enough.

Because the sync server (public or private) doesn't seem to have a "login" concept, one server will happily accept sync data from a chain that you've initially created on another. With that wrapper getting easily overwritten, there's a high risk that your brave instance starts syncing data to the public server, without any indication.

A somewhat painful workaround I've adopted is to chattr +i that wrapper, causing any brave updates to crash and fail on purpose - which then prompts me to manually update brave and re-patch the file.

An IMO adequate solution for Linux/Unix users would be if that same wrapper would, out of the box, look for command line options in /etc/brave/something and ~/.config/brave/something. Users could then add their --sync-url=... and any other custom settings in those config files. Below is an example patch. Don't know how a similar feat would be accomplished on Win/Mac.

--- /opt/brave.com/brave/brave-browser.dist     2022-11-15 03:49:50.000000000 +0200
+++ /opt/brave.com/brave/brave-browser  2022-11-18 05:42:47.065912245 +0200
@@ -1,6 +1,6 @@
 #!/bin/bash
 #
-# Copyright 2011 The Chromium Authors
+# Copyright (c) 2011 The Chromium Authors. All rights reserved.
 # Use of this source code is governed by a BSD-style license that can be
 # found in the LICENSE file.

@@ -9,6 +9,16 @@

 HERE="`dirname "$CHROME_WRAPPER"`"

+#
+#
+#
+EXTRA_FLAGS=""
+if [ -e "${HOME}/.config/brave-flags.conf" ]; then
+    while read line ; do
+       EXTRA_FLAGS="${EXTRA_FLAGS} ${line}"
+    done < "${HOME}/.config/brave-flags.conf"
+fi
+
 # We include some xdg utilities next to the binary, and we want to prefer them
 # over the system versions when we know the system versions are very old. We
 # detect whether the system xdg utilities are sufficiently new to be likely to
@@ -45,4 +55,4 @@
 exec > >(exec cat)
 exec 2> >(exec cat >&2)

-"$HERE/brave" "$@" || true
+"$HERE/brave" ${EXTRA_FLAGS} "$@" || true

[DaCHack]

IMHO the Solution should not be a wrapper that just adds the command line option because it would not easily solve the issue for mobile (particular iOS) clients. It should be a config option in the UI or at least in brave:/…

[treyperrone]

Adding my vote to this issue as well. Using MacOS/iOS primarily, but also random windows/linux boxes and even Android occasionally when needed.

[Ricent82]

Not really able to understand why the devs simply ignore this issue for so many years...

[StrangePeanut]

Very interested also. I'm guessing Brave considers this issue low priority as there already is a --sync-url argument for desktop, but is there any such workaround for Android at this moment?

[ryanhendry]

This seems like a really quick win for a privacy-focused browser. It's weird it hasn't been implemented yet. I might look into if it's possible to block the brave server URLs at a DNS level to stop data leakage.

[GanerCodes]

literally how is this not a flag on mobile yet it's the equivalent of a car without tires

[ippocratis]

For android

There is an option to enable and run custom flags with root editing /data/local/tmp/chrome-command-line

Enabling command-line in chrome://flags/#enable-command-line-on-non-rooted-devices will read the flags from that file

Just add --sync-url=192.168.1.xx/v2 to it

The file should start with an underscore (_)

Visit brave://sync-internals to verify brave is actually using the custom sync server

Brave will cry for security with a pop up message on every new tab

That is a tmp folder not sure when it gets wiped

And still it requires root (/data/local/tmp/ can be edided over adb)

Not great

But better than nothing

[GanerCodes]

well this sucks, I'd rather not have to re-install graphine just for a browser. Does anyone have a mobile fork that doesn't have to problem?

[DaCHack]

For android

There is an option to enable and run custom flags with root editing /data/local/tmp/chrome-command-line

Enabling command-line in chrome://flags/#enable-command-line-on-non-rooted-devices will read the flags from that file

Just add --sync-url=192.168.1.xx/v2 to it

The file should start with an underscore (_)

Visit brave://sync-internals to verify brave is actually using the custom sync server

Brave will cry for security with a pop up message on every new tab

That is a tmp folder not sure when it gets wiped

And still it requires root

Not great

But better than nothing

Thanks for the insight. But still cannot be the solution. Too inconvenient, insecure and Not Helping on iOS

[Kansai53]

Been watching this and waiting for a year and a half - seems like it's simply being ignored. Shame. Guess the only option is to switch to Firefox, where self-hosted sync actually works properly.

[Joshndroid]

I moved to Firefox and forgot about this to be honest..

It is a shame.

[Fmstrat]

I thought Brave was the one, but this is too big of a missing feature for mobile.

[Fmstrat]

So I've located the source point for this in Chromium. Seems Brave is using the same code for initiating the sync URL. Chromium team has this marked as aWontFix (it is Google after all). Next weekend I may take a look at this and see if I can override the in-memory sync URL variable with a custom flag. If I do, I'll start with desktop then move on to Android.

[Fmstrat]

Figured I'd take a look at this over lunch since I ran an initial Chromium build last night.

Got it working directly in Chromium: https://github.com/brave/chromium/compare/main...Fmstrat:brave-chromium:feature/custom-sync-server?expand=1

Not bad for never looking at Chromium source before ;)

So I started to port to Brave: https://github.com/brave/brave-core/compare/master...Fmstrat:brave-core:feature/custom-sync-server?expand=1

But am running into an issue where I'm not sure how to give Brave access to the syncer namespace that is part of Chromium (line 892). Are any Brave contributors following this issue that could point me in the right direction?

[Fmstrat]

Messed around this morning a bit. Found that kBraveSyncServiceStagingURL seems to be an existing override for Chromium's staging URL. So it may be easier to allow editing of the staging URL and repurpose that for custom syncing. Right now it's statically compiled into the app, but no reason I couldn't just change it to an editable field. When I get time again I'll give that a shot.

[DaCHack]

@Fmstrat Just wanted to send you a huge thanks for taking this up. I think I do this in the name of many users who are eagerly waiting for this to switch from Chrome to Brave! :)

[Fmstrat]

@DaCHack Thanks! It's slow going since it's a spare-time thing. But there has been progress:

[DaCHack]

Cool! You are working on the android Version, Right? Or is there a chance to Test it already with iOS?

[Fmstrat]

Cool! You are working on the android Version, Right? Or is there a chance to Test it already with iOS?

As with most FOSS, Android first. I don't even have an iOS device anymore, and since Brave/Chromium is native development, I probably wouldn't be the one to port ithe frontend elements over. Unfortunately someone else will need to take that on (that is if there are any community developers who use iOS).

[GanerCodes]

@fastman Any new updates 👀

[Fmstrat]

@GanerCodes

Decided to just push what I had done last week real quick while taking a tea break. The UI is complete and the custom URL saves inside the preferences: https://github.com/Fmstrat/brave-core/compare/master...Fmstrat:brave-core:feature/custom-brave-sync-server

The big problem I've run into now is that Brave is only altering the sync server used by Chromium at startup: https://github.com/brave/brave-core/blob/d167fcf7df2e0a7f3d5aa5ae375c4de4a9eedb5f/chromium_src/chrome/app/chrome_main_delegate.cc#L73

This is why the only way to use the custom sync now is via root and altering the startup command. It mirrors what is done on desktop with the CLI option (which is actually a part of Chromium). I'm new to Brave dev, and as stated only have a small amount of time to focus on this every now and then, so I haven't had the time to figure out how to get access to Chromium's syncer::kSyncServiceURL variable while the app is running.

I may need to shift gears and go back to a flags method but somehow alert the user that a restart is required for changing that value. I'll need to look at how the flag for swapping to the staging URL works, likely.

[Fmstrat]

Flag method was actually fairly trivial. It required a restart by default: https://github.com/Fmstrat/brave-core/compare/master...Fmstrat:brave-core:feature/custom-flag-sync-server

Only problem now is the sync service doesn't seem to be hitting the server. It may be a URL permissions thing, so will have to investigate another time (Could also very easily be me not using the default value correctly, modeled after another field, and only spent 15 minutes on this, though I did test hard-coding the URL I'm using).

[Fmstrat]

To start: I was able to get Android and Desktop working together with a self-hosted server, and wrote up directions and supplied a Docker Compose file in a PR here: https://github.com/brave/go-sync/pull/168/files

The instructions point out that the command line options on Android may not survive a reboot, but so far on my emulator they have.

Over the weekend I got a bit further along with using flags, but was unable to complete the connection between overriding in the same location as the CLI and the flags themselves. So, work is ongoing.

[ippocratis]

The instructions point out that the command line options on Android may not survive a reboot, but so far on my emulator they have.

Been selfhosting the go-sync server since June on various android devices

The sync URL under /data/local/tmp/chrome-command-line survives reboots and OS updates fine

Would be nice if brave could read from /data/local/chrome-command-line too though as chromium do .

Maybe a sidenote in your pull request is also needed that if enabling the command line feature flag will greet them with an anr On every new tab

"You are using am umsuported feature flag commamdlineonnonrooted stability and security will suffer"

[Fmstrat]

Would be nice if brave could read from /data/local/ too though as chromium do .

Could you explain this?

[ippocratis]

Would be nice if brave could read from /data/local/ too though as chromium do .

Could you explain this?

Chromium can read feature flags from /data/local/chrome-command-line

All chromium browser do

Brave is not

I'm rooted and tried it

[czadikem]

I am very intreasted in this and will be sure to try this soon. Hopefully we can fix this, so it is not so hard to implement.