search

microsoft / vscode

Visual Studio Code
https://code.visualstudio.com
MIT License
160.75k stars 28.17k forks source link

Self-signed certificate error when installing Python support in WSL in spite of custom root certificate correctly installed in WSL #131836

Open srinathh opened 2 years ago

srinathh commented 2 years ago

Does this issue occur when all extensions are disabled?: Yes/No

My company uses an SSL inspection on company devices and provides a custom root CA certificate. In Windows, this is already pre-installed in the system certificate store and VS Code works fine in Windows where there are no problems installing & using Python language support. For WSL, the certificates & a process to install them in the system certificate store are provided (below) However, when trying to setup Python Language support in WSL I get a self-signed certificate error in spite of following the process to install the custom root certificates in WSL.

Steps to Reproduce:

  1. Copy custom certificates to WSL CA-Certificates folder 
    sudo cp *.crt /usr/local/share/ca-certificates/
  2. Install 
    sudo update-ca-certificates
  3. Launch VS Code and connect to WSL project
  4. Try to install Python support in WSL
  5. Get error - self signed certificate in certificate chain
srinathh commented 2 years ago

Hello! I've developed a work around.

The core issue here seems to be that the certificate store passed by NodeJS running the back-end server to the OpenSSL library it uses to deal with encryption is not the linux system certificate store updated by update-ca-certificates. However, NodeJS has an environment variable configuration option NODE_EXTRA_CA_CERTS in which we can pass it the custom certificate. We can setup this environment variable in ~/.vscode-server/server-env-setup in WSL.

Something like this works

echo "export NODE_EXTRA_CA_CERTS=<PATH TO SELF SIGNED ROOT>" >>  ~/.vscode-server/server-env-setup

However, ideally the solution should be to have NodeJS use the system certificate store

aeschli commented 2 years ago

@chrmarti Are you familiar with this?

chrmarti commented 2 years ago

We read /etc/ssl/certs/ca-certificates.crt and /etc/ssl/certs/ca-bundle.crt and add these to all requests made with Node.js' https module from the remote extension host.

Maybe the failing request is made somewhere else. Could you attach a screenshot from when the error occurs?

srinathh commented 2 years ago

Hello - here's the relevant failure screenshot. image

I also tried installing the extension manually & got the same error.

Installing extensions...
self signed certificate in certificate chain
Failed Installing Extensions: file:///home/<my home dir>/Downloads/ms-python.python-2021.8.1159798656.vsix
(node:394) UnhandledPromiseRejectionWarning: Canceled: Canceled
    at n (/home/<my home dir>/.vscode-server/bin/3866c3553be8b268c8a7f8c0482c0c0177aa8bfa/out/vs/server/remoteExtensionHostAgent.js:8:1157)
    at /home/<my home dir>/.vscode-server/bin/3866c3553be8b268c8a7f8c0482c0c0177aa8bfa/out/vs/server/remoteExtensionHostAgent.js:68:11867
    at b.fire (/home/<my home dir>/.vscode-server/bin/3866c3553be8b268c8a7f8c0482c0c0177aa8bfa/out/vs/server/remoteExtensionHostAgent.js:60:1712)
    at D.cancel (/home/<my home dir>/.vscode-server/bin/3866c3553be8b268c8a7f8c0482c0c0177aa8bfa/out/vs/server/remoteExtensionHostAgent.js:60:5929)
    at I.cancel (/home/<my home dir>/.vscode-server/bin/3866c3553be8b268c8a7f8c0482c0c0177aa8bfa/out/vs/server/remoteExtensionHostAgent.js:60:6470)
    at Object.cancel (/home/<my home dir>/.vscode-server/bin/3866c3553be8b268c8a7f8c0482c0c0177aa8bfa/out/vs/server/remoteExtensionHostAgent.js:68:11996)
    at Q.cancel (/home/<my home dir>/.vscode-server/bin/3866c3553be8b268c8a7f8c0482c0c0177aa8bfa/out/vs/server/remoteExtensionHostAgent.js:99:17267)
    at /home/<my home dir>/.vscode-server/bin/3866c3553be8b268c8a7f8c0482c0c0177aa8bfa/out/vs/server/remoteExtensionHostAgent.js:99:8033
    at Array.forEach (<anonymous>)
    at k.installExtension (/home/<my home dir>/.vscode-server/bin/3866c3553be8b268c8a7f8c0482c0c0177aa8bfa/out/vs/server/remoteExtensionHostAgent.js:99:8011)
    at processTicksAndRejections (internal/process/task_queues.js:93:5)
    at async d.installVSIX (/home/<my home dir>/.vscode-server/bin/3866c3553be8b268c8a7f8c0482c0c0177aa8bfa/out/vs/server/remoteExtensionHostAgent.js:77:260)
    at async /home/<my home dir>/.vscode-server/bin/3866c3553be8b268c8a7f8c0482c0c0177aa8bfa/out/vs/server/remoteExtensionHostAgent.js:76:40837
    at async Promise.all (index 0)
    at async d.installExtensions (/home/<my home dir>/.vscode-server/bin/3866c3553be8b268c8a7f8c0482c0c0177aa8bfa/out/vs/server/remoteExtensionHostAgent.js:76:40791)
    at async /home/<my home dir>/.vscode-server/bin/3866c3553be8b268c8a7f8c0482c0c0177aa8bfa/out/vs/server/remoteExtensionHostAgent.js:116:110599
    at async k.run (/home/<my home dir>/.vscode-server/bin/3866c3553be8b268c8a7f8c0482c0c0177aa8bfa/out/vs/server/remoteExtensionHostAgent.js:116:110463)
    at async Y (/home/<my home dir>/.vscode-server/bin/3866c3553be8b268c8a7f8c0482c0c0177aa8bfa/out/vs/server/remoteExtensionHostAgent.js:116:112805)

Additionally, based on your comment - I confirmed using grep that the update-ca-certificates does indeed successfully add my custom root certificate to /etc/ssl/certs/ca-certificates.crt & it's there

grep <part of custom root cert text> /etc/ssl/certs/ca-certificates.crt

So net net, I think somewhere in VSCode when extensions are being downloaded, the certs in /etc/ssl/certs/ca-certificates.crt are not actually being passed through. However, if I explicitly set the environment variable NODE_EXTRA_CA_CERTS when starting VSCode in ~/.vscode-server/server-env-setup it does work.

srinathh commented 2 years ago

Hi @chrmarti @aeschli - any luck identifying where the issue might be? Any other outputs I can pull out or test to help identify? We've now confirmed multiple folks in my company faced this issue but have been able to use the work-around and we've documented it internally.

chrmarti commented 2 years ago

@sandy081 Are you downloading the VSIX using the Node.js https module in the renderer/shared process? Did this change from using the Chromium network stack?

sandy081 commented 2 years ago

I use the request service to download and it seems this is the request service that is being registered in renderer -

https://github.com/microsoft/vscode-distro/blob/731402dce225ce25d42a7b173361650a54bf52ad/src/vs/workbench/services/request/electron-sandbox/requestService.ts#L13-L13

It seems this uses chrome stack.

aeschli commented 2 years ago

Looks like various issues here:

gbeal-smsc commented 2 years ago

Hello! I've developed a work around.

The core issue here seems to be that the certificate store passed by NodeJS running the back-end server to the OpenSSL library it uses to deal with encryption is not the linux system certificate store updated by update-ca-certificates. However, NodeJS has an environment variable configuration option NODE_EXTRA_CA_CERTS in which we can pass it the custom certificate. We can setup this environment variable in ~/.vscode-server/server-env-setup in WSL.

Something like this works

echo "export NODE_EXTRA_CA_CERTS=<PATH TO SELF SIGNED ROOT>" >>  ~/.vscode-server/server-env-setup

However, ideally the solution should be to have NodeJS use the system certificate store

You are the hero I needed, thanks for this!

shsuman commented 2 years ago

One of our customers is also impacted by the same issue when connecting to Azure ML Compute instances through VSCode. Azure ML - Remote extension tried to install the Python extension as part of setting up the VSCode server and that's where they get this error: [2021-10-04 17:03:27.801] [renderer6] [error] Error while installing 'ms-python.python' extension in the remote server. self signed certificate in certificate chain

J0F3 commented 2 years ago

@chrmarti and @sandy081 Can you again look into that. It seems to be pop up again with the 1.62 version of VS Code (tough with all extension, not just the python). I have tested it by rolling back to 1.61.2 and then I can install all extensions again in WSL. But then as soon as update to 1.62 the installation of any extension in WSL fails. (The version "Remote - WSL" extensions is in both cases still the same same version - 0.58.5) When looking into the log it is quite obvious what happens different:

In version 1.61.2 the cert error also happens but then it automatically downloads the extensions locally instead (line 1 & 2):

[2021-11-05 14:12:42.022] [renderer1] [error] Error while installing 'dbaeumer.vscode-eslint' extension in the remote server. self signed certificate in certificate chain**
[2021-11-05 14:12:42.022] [renderer1] [info] Downloading the 'dbaeumer.vscode-eslint' extension locally and install
[2021-11-05 14:12:42.067] [renderer1] [info] Downloaded extension: dbaeumer.vscode-eslint /c:/Users/j0f3/AppData/Local/Temp/1e84c682-6562-432a-8e63-78d8303469ef
[2021-11-05 14:12:43.133] [renderer1] [info] Successfully installed 'dbaeumer.vscode-eslint' extension

But now with version 1.62 the process just stops at the cert error:

[2021-11-05 14:14:53.702] [renderer1] [error] ["Error: self signed certificate in certificate chain","    at TLSSocket.onConnectSecure (_tls_wrap.js:1497:34)","    at TLSSocket.emit (events.js:315:20)","    at TLSSocket._finishInit (_tls_wrap.js:932:8)","    at TLSWrap.ssl.onhandshakedone (_tls_wrap.js:706:12)"]
[2021-11-05 14:14:53.714] [renderer1] [error] self signed certificate in certificate chain: Error: self signed certificate in certificate chain
    at TLSSocket.onConnectSecure (_tls_wrap.js:1497:34)
    at TLSSocket.emit (events.js:315:20)
    at TLSSocket._finishInit (_tls_wrap.js:932:8)
    at TLSWrap.ssl.onhandshakedone (_tls_wrap.js:706:12)

I can workaround it for now by settings the remote.downloadExtensionsLocally setting to true or by using the method @gbeal-smsc has describe above. But while looking at the solution from @gbeal-smsc I think it would be the best if the vscode-server would somehow automatically inherit the NODE_EXTRA_CA_CERTS environment variable if it is set in WSL already. Because of our corporate Proxy with TLS-Inspection I have to set the NODE_EXTRA_CA_CERTS environment variable anyway in WSL but it is then not very intuitive that the vscode-server does not know anything about it by default.

So not sure which one would actually be the "correct" solution but I think either the extension installation process should automatically fail back again to local download (as it was in 1.61.2) or the vscode-server should automatically pickup the NODE_EXTRA_CA_CERTS environment variable if it is set in WSL. But just fail with the cert error and leave the user with the problem, as it is now in 1.62, is IHMO the worst option. 😉

Thanks!

sandy081 commented 2 years ago

In 1.62.0, I have made retry downloading (fallback) from local bit more strict and is triggered only when we see specific errors as it does not makes sense to retry if it failed for some obvious reasons.

https://github.com/microsoft/vscode/blob/5e0d7da4072cb1b7fe3be450920b0ce2f7290da5/src/vs/workbench/services/extensionManagement/electron-sandbox/remoteExtensionManagementService.ts#L61-L72

I would be interested to know what error (name) is thrown in your case. So I added a debug statement for that. Can you please try with tomorrow's insiders and let me know that.

For those users, who have connectivity issues on remote, there is a setting you can use to always download the extension locally - remote.downloadExtensionsLocally. I would recommend to set this setting instead of relying on fallback we have.

J0F3 commented 2 years ago

In 1.62.0, I have made retry downloading (fallback) from local bit more strict and is triggered only when we see specific errors as it does not makes sense to retry if it failed for some obvious reasons.

Ah yes that makes sense.

I would be interested to know what error (name) is thrown in your case. So I added a debug statement for that. Can you please try with tomorrow's insiders and let me know that.

Sure, I will do that and keep you posted. 👍

J0F3 commented 2 years ago

I have just tested with the insider version (2021-11-10T07:59:05.913Z, Commit: bef4dba21f68bbdb8d9f08fe04b2e1858597722b) but I am afraid that there is not really more to see in the log:

[2021-11-10 12:15:38.172] [renderer1] [debug] Remote Install Error Name Error
[2021-11-10 12:15:38.202] [renderer1] [error] ["Error: self signed certificate in certificate chain","    at TLSSocket.onConnectSecure (_tls_wrap.js:1497:34)","    at TLSSocket.emit (events.js:315:20)","    at TLSSocket._finishInit (_tls_wrap.js:932:8)","    at TLSWrap.ssl.onhandshakedone (_tls_wrap.js:706:12)"]
[2021-11-10 12:15:38.230] [renderer1] [error] self signed certificate in certificate chain: Error: self signed certificate in certificate chain
    at TLSSocket.onConnectSecure (_tls_wrap.js:1497:34)
    at TLSSocket.emit (events.js:315:20)
    at TLSSocket._finishInit (_tls_wrap.js:932:8)
    at TLSWrap.ssl.onhandshakedone (_tls_wrap.js:706:12)

Further I have noticed also problem installing extensions during dev container build and startup. I have a dev container setup wich should install the ms-vscode.powershell when the container is created. But after the container is created and running the extension is not installed. In the Remote Server Log i see the following:

[2021-11-10 11:06:19.699] [remoteagent] [info] 

*
* Visual Studio Code Server
*
* Reminder: You may only use this software with Visual Studio family products,
* as described in the license https://aka.ms/vscode-remote/license
*

[2021-11-10 11:06:19.699] [remoteagent] [info] Extension host agent started.
[2021-11-10 11:06:21.679] [remoteagent] [error] Error: self signed certificate in certificate chain
    at TLSSocket.onConnectSecure (_tls_wrap.js:1497:34)
    at TLSSocket.emit (events.js:315:20)
    at TLSSocket._finishInit (_tls_wrap.js:932:8)
    at TLSWrap.ssl.onhandshakedone (_tls_wrap.js:706:12)
[2021-11-10 11:06:21.787] [remoteagent] [info] [::ffff:127.0.0.1][5b720351][ManagementConnection] New connection established.

So it seems to be because of the same error as in WSL. That this is also when I have "remote.downloadExtensionsLocally": true in my settings. Installing the extension then manually after the container has started works. But it is not installed automatically during container creation.

sandy081 commented 2 years ago

Thanks for the logs @J0F3. I got what I am looking for

[2021-11-10 12:15:38.172] [renderer1] [debug] Remote Install Error Name Error

It seems this is an error that I am not handling properly so that the fallback can be triggered - I will be fixing this here - https://github.com/microsoft/vscode/issues/136710, So please follow up there.

So it seems to be because of the same error as in WSL. That this is also when I have "remote.downloadExtensionsLocally": true in my settings. Installing the extension then manually after the container has started works. But it is not installed automatically during container creation.

While container is being created, extensions are downloaded and installed on remote server/container. Since the VS Code client is not yet connected to server, it is not possible to download on client and install on remote. CC @chrmarti

J0F3 commented 2 years ago

It seems this is an error that I am not handling properly so that the fallback can be triggered - I will be fixing this here - #136710, So please follow up there.

Perfect! Thx!👍

So it seems to be because of the same error as in WSL. That this is also when I have "remote.downloadExtensionsLocally": true in my settings. Installing the extension then manually after the container has started works. But it is not installed automatically during container creation.

While container is being created, extensions are downloaded and installed on remote server/container. Since the VS Code client is not yet connected to server, it is not possible to download on client and install on remote. CC @chrmarti

Aha that makes sense. Thanks to your explanation I found what the problem was here. The container did not use the custom Root CA Certs of our company firewall. When I correctly define the NODE_EXTRA_CA_CERTS environment variable in my dev container configuration then the extension get installed automatically again. This one was then more a missing config from my side.

Thank you for your help!

megakid commented 4 months ago

Had this issue with my new WSL2 environment.

We add two self signed root CAs to our trusted root certificates (via update-ca-certificates on the WSL side and added to Windows certificate store on the Windows side). I fixed the error by adding this file:

❯ cat ~/.vscode-server/server-env-setup export NODE_EXTRA_CA_CERTS=/etc/ssl/certs/ca-certificates.crt

❯ lsb_release -a No LSB modules are available. Distributor ID: Ubuntu Description: Ubuntu 22.04.3 LTS Release: 22.04 Codename: jammy

chrmarti commented 4 months ago

VS Code 1.85 and later should automatically load the remote OS certificates for remote extensions. For WSL and local dev containers the local OS certificates are also loaded.

@megakid Have you tried with VS Code 1.85 or later without setting NODE_EXTRA_CA_CERTS?

michaelteo commented 3 months ago

VS Code 1.85 and later should automatically load the remote OS certificates for remote extensions. For WSL and local dev containers the local OS certificates are also loaded.

@megakid Have you tried with VS Code 1.85 or later without setting NODE_EXTRA_CA_CERTS?

I needed to do the same workaround to install Jupyter Extension on WSL2 Ubuntu22.04. VSCode version is 1.87.2

mrhorvath commented 3 months ago

VS Code 1.85 and later should automatically load the remote OS certificates for remote extensions. For WSL and local dev containers the local OS certificates are also loaded. @megakid Have you tried with VS Code 1.85 or later without setting NODE_EXTRA_CA_CERTS?

I needed to do the same workaround to install Jupyter Extension on WSL2 Ubuntu22.04. VSCode version is 1.87.2

Yes, I had the exact same issue installing the Jupyter Extension (Id: ms-toolsai.jupyter) extension in WSL2 Ubuntu22.04.

Version: 1.87.2 (user setup) Commit: 863d2581ecda6849923a2118d93a088b0745d9d6 Date: 2024-03-08T15:20:17.278Z Electron: 27.3.2 ElectronBuildId: 26836302 Chromium: 118.0.5993.159 Node.js: 18.17.1 V8: 11.8.172.18-electron.0 OS: Windows_NT x64 10.0.19045

Adding the environmental variable NODE_EXTRA_CA_CERTS to ~/.vscode-server/server-env-setup fixed the issue for me for now.

jonbackhaus commented 1 month ago

VS Code 1.85 and later should automatically load the remote OS certificates for remote extensions. For WSL and local dev containers the local OS certificates are also loaded.

@megakid Have you tried with VS Code 1.85 or later without setting NODE_EXTRA_CA_CERTS?

Just to connect the dots here, I started at #144625 and eventually found this thread. I'm running a different configuration -- remote Linux over SSH into a (remote) dev container on the Linux host -- but ran into a similar problem during container build.

We use a custom container, so I added the NODE_EXTRA_CA_CERTS variable to the container build file and set the path to the system certificate bundle. (We use a Debian base, so the configuration is NODE_EXTRA_CA_CERTS="/etc/ssl/certs/ca-certificates.crt". This solved the problem for us, just in case anyone else runs into a similar problem and configuration.

Note: we originally tried adding this configuration to our devcontainer.json specification, which did not work.