Error while starting Sync: Server returned 431

[demyte]

Issue Type: Bug

When I try to enable the new insider feature for sync I do the following

STEPS:

• Sync: Turn on Sync
• Select "Turn on Sync" from preview warning dialog
• Choose to sync all and hit "Turn On"
• Login with a Work account

EXPECTED RESULT

• Sync turns on

ACTUAL RESULT

• Error while starting Sync: Server returned 431

VS Code version: Code - Insiders 1.44.0-insider (efd7548df65de8e775015b67863190e6bd4934fc, 2020-03-13T06:05:25.782Z) OS version: Windows_NT x64 10.0.19033

System Info

|Item|Value| |---|---| |CPUs|Intel(R) Core(TM) i7-6700K CPU @ 4.00GHz (8 x 4008)| |GPU Status|2d_canvas: enabled
flash_3d: enabled
flash_stage3d: enabled
flash_stage3d_baseline: enabled
gpu_compositing: enabled
multiple_raster_threads: enabled_on
oop_rasterization: disabled_off
protected_video_decode: unavailable_off
rasterization: enabled
skia_renderer: disabled_off_ok
video_decode: enabled
viz_display_compositor: enabled_on
viz_hit_test_surface_layer: disabled_off_ok
webgl: enabled
webgl2: enabled| |Load (avg)|undefined| |Memory (System)|31.88GB (15.11GB free)| |Process Argv|| |Screen Reader|no| |VM|0%|

Extensions (5)

Extension|Author (truncated)|Version ---|---|--- azure-pipelines|ms-|1.165.1 remote-wsl|ms-|0.42.3 azure-account|ms-|0.8.8 powershell|ms-|2020.3.0 azure-iot-toolkit|vsc|2.16.0

[vscodebot[bot]]

(Experimental duplicate detection) Thanks for submitting this issue. Please also check if it is already covered by an existing one, like:

• Setting sync error messages language (#89705)
• Expect some error message when sync fails and for sync to turn off (#89485)
• Error notification when turning off sync while signed out (#90916)
• No way to change what is synced while sync is enabled (#91504)
• Sync extensions enablement (#86916)

[Tyriar]

@sandy081 431 Request Header Fields Too Large? Are you aware of sending any large headers other than authorization?

[sandy081]

@Tyriar No.

[Tyriar]

8kb is the hard limit in node https://nodejs.org/api/errors.html#errors_hpe_header_overflow

[Tyriar]

@demyte can you run this command in the command palette (F1)

Then open the network tab and then go back to the VS Code window and try turn on sync? This should show the headers being sent:

I'm interested in which header entries there are and how many characters there are in total. Note that the authorization header contains sensitive information and you should not paste it here.

[demyte]

Hi @Tyriar,

Here is what that is showing me:

[Tyriar]

@demyte are there any below authorization?

[demyte]

@Tyriar they are all showing there, except user agent.

I chopped out the auth part and moved the ones below up in the image

[phealy]

I'm having the same problem when enabling sync after signing in to my Microsoft account (literally @microsoft.com, I'm an FTE).

According to Fiddler, it is exceeding 8k of headers:

Request Count:   1
Bytes Sent:      8,624      (headers:8,624; body:0)
Bytes Received:  121        (headers:121; body:0)

The request is pretty simple aside from a giant 4kb bearer token:

GET /v1/manifest HTTP/1.1
Host: vscode-sync.trafficmanager.net
Connection: keep-alive
authorization: Bearer <4096 character token removed>
X-Sync-Client-Id: 1.44.0-insider
X-Sync-Machine-Id: xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Code-Insiders/1.44.0-insider Chrome/78.0.3904.130 Electron/7.1.11 Safari/537.36
Content-Type: application/json
Accept: */*
Sec-Fetch-Site: cross-site
Sec-Fetch-Mode: cors
Accept-Encoding: gzip, deflate, br
Accept-Language: en-US

I decoded the JWT, and most of the length is the 122 guids in the groups list. If you can't increase the allowed header size, perhaps the AAD app registration could have groupMembershipClaims set to null so that it's not getting the groups in the token, if that wouldn't affect anything else using the same app registration?

[sandy081]

@Tyriar Any updates?

[Tyriar]

@RMacfarlane this is happening because the auth token is enormous.

[Tyriar]

Unassigning myself and @sandy081 as this is an auth issue.

[rmcolbert]

I have the same issue (non-MS FTE) ... my AAD auth token is even larger. Bearer token itself is 9170 bytes. @Tyriar Is there a request w/ OpenJS to make max header size in http_parser a user configurable option (like you can on most other web server platforms)?

[linickx]

Does the team need any further debugs?

I can provide comparison logs have access to 2x o365 tenants, one personal and one corporate, the personal one works and the company one fails with this 431 error.

Mac User...

mbp:~ $ code -v
1.48.2
a0479759d6e9ea56afa657e454193f72aef85bd0
x64
mbp:~ $ sw_vers
ProductName:    Mac OS X
ProductVersion: 10.14.6
BuildVersion:   18G5033
mbp:~ $ 

[rkeithhill]

Same problem here trying to sync using my corporate M365 account:

Request Count:   1
Bytes Sent:      12,388     (headers:12,388; body:0)
Bytes Received:  104        (headers:104; body:0)

My bearer token length is 11602. I'd really like to be able to sync my work settings with my corp account because my work settings are quite different (bunch of Atlassian tools settings) from my home settings (more OSS / GitHub oriented).

[MewX]

Same here, retried 3 times and finally worked.

[tiagoh]

Any updates on this issue ? I can't find a way to sync my VS Code settings at all due to this Error.

[phealy]

Any updates on this issue ? I can't find a way to sync my VS Code settings at all due to this Error.

I'm still using the Settings Sync extension since that goes to a GitHub gist - though I'd prefer to use the stock functionality.

[charltonstanley]

I also have the same issue when using my corp. MS account. I could try using my personal outlook account but something tells me that my org would be significantly less than impressed me doing with that...

Version: 1.50.1 (user setup) Commit: d2e414d9e4239a252d1ab117bd7067f125afd80a Date: 2020-10-13T15:06:15.712Z Electron: 9.2.1 Chrome: 83.0.4103.122 Node.js: 12.14.1 V8: 8.3.110.13-electron.0 OS: Windows_NT x64 10.0.18363

[JvDrunen]

Any update? not working here:

Version: 1.52.1 (system setup) Commit: ea3859d4ba2f3e577a159bc91e3074c5d85c0523 Date: 2020-12-16T16:34:46.910Z (1 mo ago) Electron: 9.3.5 Chrome: 83.0.4103.122 Node.js: 12.14.1 V8: 8.3.110.13-electron.0 OS: Windows_NT x64 10.0.19042

[tonka3000]

Any updates? I got a new machine and settings sync does not work at all.

Version: 1.52.1 (user setup) Commit: ea3859d4ba2f3e577a159bc91e3074c5d85c0523 Date: 2020-12-16T16:34:46.910Z Electron: 9.3.5 Chrome: 83.0.4103.122 Node.js: 12.14.1 V8: 8.3.110.13-electron.0 OS: Windows_NT x64 10.0.17763

[Dongbumlee]

same issue from here.

[tonka3000]

@RMacfarlane Any chance to get that fixed anytime soon?

Maybe cut of the after 8kb header. Not a great solution but maybe it could work (could create clashes of course).

Or is there any possibility to make the guids shorter on the client side?

[tmachqy]

Yep, running into this issue on multiple AAD accounts that were previously working. Not fun when I went to rebuild one of my machines.

Version: 1.54.3 (user setup)

[MietekW]

Getting the same error when turning on the sync, logged on with my company account.

[2021-04-01 10:03:28.870] [settingssync] [info] Request failed {"url":"https://vscode-sync.trafficmanager.net/v1/manifest","status":431,"execution-id":"c3b7f01a-05b9-47c1-a002-45d1d4a665a4"}

Code -v

1.55.0 c185983a683d14c396952dd432459097bc7f757f x64

[TaylerUva]

same issue

[greyknight79]

Seeing the same issue with my O365 account.

[2021-04-14 08:28:07.099] [settingssync] [info] Request failed {"url":"https://vscode-sync.trafficmanager.net/v1/manifest","status":431,"execution-id":"0e78b554-9cf2-44c7-8f74-2f5a7cbf831c"}

Code -v 1.55.2 3c4e3df9e89829dce27b7b5c24508306b151f30d x64

[thuandt]

I have same issue since last week with my O365 account

[osbornepb]

Same issue

[dansmitt]

@RMacfarlane any updates?

[TylerLeonhardt]

It looks like the ability to change the max header size was added in Node.js 10 and the default was increased to 16kb in Node.js 13.

https://nodejs.org/api/cli.html#cli_max_http_header_size_size

Since we don't use node 13 yet because of the version of Electron we're on, I think we can probably set our default to 16kb for now since that is the default in future node versions.

With all that said...I do wonder what we can do to decrease the size of this payload because 12kb is a LOT to send over in pretty much every request...if anyone has any AAD insights I'm all ears.

[tonka3000]

@TylerLeonhardt is there a way to get the token on my VSCode instance? I would like to check why my token is so long. I'm also not sure if the token is opaque or not? Would be interesting to see why the token is so long.

I'm my case sync works on 2 machines and on one not. So for me the question is: what is included in the token. Does VSCode provide extra data for the AAD token generation or is the token directly from the AAD? Because the sync needs to identify the user as well as the exact machine. I think that this IDs are pretty long by default especially in enterprises. Maybe you have/can add some telemetry data about the length of the token, maybe they are often near the limits.

Greetings Michael

[TylerLeonhardt]

@tonka3000 you can probably follow these steps (which I haven't tried yet myself):

https://github.com/microsoft/vscode/issues/92705#issuecomment-600641216

If one of your machines is running Windows.

[tonka3000]

I will try it. All 3 machines are windows machines.

I think I will have the same group id problem as mention in the linked comment. The mitigation (deactivate group IDs at the app level) from the comment would make sense, what is your opinion on that topic? Enterprise will have this problem a lot.

[phealy]

@TylerLeonhardt setting the app registration to not get a groups scope is something that has to be done by the team maintaining the VS Code sync server; it's an Azure Active Directory app registration setting. It can't be changed client side. Setting that would basically tell Azure AD "I don't care about the groups scope, so don't even send it".

[TylerLeonhardt]

@phealy I'm the new owner of the authentication stack and a member of the VS Code team 😁

I'll try to look at the first party app portal to see if I can find such a setting but it's slightly different than the Azure portal.

If you know exactly how I should do this, hit me up on email/Teams!

[phealy]

I know how to do it in AAD and I think I sent you the correct docs via Teams for doing it in the 1P portal, so hopefully that will work! I was more letting @tonka3000 know that it wasn't anything that could be tried on the client.

I got myself out of a bunch of groups and no longer hit the issue, so it's harder to reproduce, but I can at least validate the tokens to see if they're returning a group scope.

[TylerLeonhardt]

Can't believe I missed this but this is because Settings Sync server isn't on Node >13 yet. I'll move this issue over there and will report back here when that work is done.

[timovp]

Any update / ETA yet on this yet? I'm facing the same issue as others above with corporate AAD here.

[TylerLeonhardt]

The work in the settings sync server is assigned for this milestone.

[TylerLeonhardt]

The server has been updated to Node 14.

[tonka3000]

@TylerLeonhardt Thank you. Works perfect now 🎉🎉🎉