microsoft / vscode-tools-for-ai

Azure Machine Learning for Visual Studio Code, previously called Visual Studio Code Tools for AI, is an extension to easily build, train, and deploy machine learning models to the cloud or the edge with Azure Machine Learning service.
Other
321 stars 91 forks source link

Not able to connect to VS code web and VS code desktop extension in AML #2282

Closed marijnbaar closed 4 months ago

marijnbaar commented 5 months ago

Hi there, I am experiencing an issue with VS code web and VS code desktop extension in AML. When I try to connect, I get the message "maximal tries exhausted, could not install VS Code Server".

Does this occur consistently: yes.

Repro steps:

  1. Start an AML instance.
  2. Open VS Code web or Desktop - keep getting the message "maximal tries exhausted, could not install VS Code Server. Downloading Vs Code server failed. Please try again.
  3. When clicking on "Kill VS Code Server and try again" the same problem keeps returning.

Error message: Downloading VS Code server failed. Please try again later. Error type: VSCodeServerInstallerError Error code: 532

Version: 0.44.0 Browser Name: Chrome Browser Version: 121.0.0.0 OS: OS X 10.15.7 64-bit Product: Visual Studio Code Product Version: 1.85.2 Language: nl-nl

Call Stack ``` Xn.tryInstall webOpener.js:2:764696 async Xt.retries webOpener.js:2:751332 async Xt webOpener.js:2:722683 async Xt webOpener.js:2:722969 async Xt webOpener.js:2:722969 async Xt webOpener.js:2:722969 async Xt webOpener.js:2:722969 async Xt webOpener.js:2:722969 async Xt webOpener.js:2:722969 async Xt webOpener.js:2:722969 ```
shsuman commented 5 months ago

@marijnbaar Is your workspace behind a vnet with public network access disabled ?

marijnbaar commented 5 months ago

@shsuman Yes, it is.

shsuman commented 5 months ago

@marijnbaar Please whitelist the following updated list of endpoints for the connection: https://code.visualstudio.com/docs/setup/network

redonovan commented 5 months ago

I have the same issue with VS Code Desktop as of this morning GMT - it was working yesterday.
ssh from a WSL Ubuntu terminal doesn't work either, which it did yesterday. Meanwhile the ml.azure.com dashboard says everything is fine with my running instance. I'm at home - Windows 11 Firewall says VSCode and WSL have both public and private access. Where do we whitelist those endpoints? Thanks.

VSCode : "Maximum retries exhausted. Could not install VS Code server on rob3: Server installation process already in progress - waiting and retrying"

ssh azureuser@172.NNN.NN.128 -p 50000 kex_exchange_identification: Connection closed by remote host Connection closed by 172.NNN.NN.128 port 50000

marijnbaar commented 5 months ago

@shsuman Is there something that has changed in this list? Because we had whitelisted all the necessary endpoints, and it was working before - until one week ago, approximately.

shsuman commented 5 months ago

@marijnbaar Yes, VSCode has changed its download CDN's endpoints. We are in the process of updating our documentation with all the necessary endpoints. For the time being, please use VSCode's documentation that I pasted above accordingly.

shsuman commented 5 months ago

@redonovan Is your workspace behind VNET with public network access being disabled ?

redonovan commented 5 months ago

If you mean our "workspace" on ml.azure.com then as far as I know the answer is no to your VNET question: On the details page of my running instance there's a section "Virtual network/subnet" that displays "--". The instance has a public IP address and until yesterday I could ssh in, so I suspect public network access is/was not disabled. The instance was created 13 Dec last year, if that's relevant.

shsuman commented 5 months ago

@redonovan Are you able to access the terminal of the CI from 'ml.azure.com' ?

Also, for the SSH problem, please contact Azure ML support as we are not equipped to help with that.

redonovan commented 5 months ago

I can! I was able to get in through the Terminal link on the Details page on ml.azure.com Where I am told "Your instance's OS image version (Linux 23.10.24) isn’t the most recent (Linux [23.12.08])", with instructions on how to upgrade. I guess that's it - some update my instance didn't get that made both VSCode and ssh access stop working. Hmm, looks like it's worse than that : my job was running in screen, which is still there, and it looks like everything froze at or shortly after 19:01 GMT on Jan 29th. The recommended upgrade is to recreate your instance, so I'll go and do that. I suspect that will fix the access problems - if I still can't get in I'll file a new issue. Thanks.

marijnbaar commented 5 months ago

Hi there @shsuman, I’ve added all the endpoints, but this doesn’t solve my issue.

shsuman commented 5 months ago

@marijnbaar Can you access the terminal of the Compute instance in the browser and try to run the following command:

wget --connect-timeout=7 --dns-timeout=7 -O -v -d vscode-server.tar.gz https://update.code.visualstudio.com/commit:8b3775030ed1a69b13e4f4c628c612102e30a681/server-linux-x64/stable

marijnbaar commented 5 months ago

Done that. This is the output:

Created socket 4. Releasing 0x000055b30eedf9b0 (new refcount 1). Initiating SSL handshake. Handshake successful; connected socket 4 to SSL handle 0x000055b30eee0ff0 certificate: subject: CN=update.code.visualstudio.com,O=Microsoft Corporation,L=Redmond,ST=WA,C=US issuer: CN=Microsoft Azure RSA TLS Issuing CA 08,O=Microsoft Corporation,C=US X509 certificate successfully verified and matches host update.code.visualstudio.com

---request begin--- GET /commit:8b3775030ed1a69b13e4f4c628c612102e30a681/server-linux-x64/stable HTTP/1.1 User-Agent: Wget/1.20.3 (linux-gnu) Accept: / Accept-Encoding: identity Host: update.code.visualstudio.com Connection: Keep-Alive

---request end--- HTTP request sent, awaiting response... ---response begin--- HTTP/1.1 302 Found Date: Thu, 01 Feb 2024 08:57:01 GMT Content-Type: text/plain; charset=utf-8 Content-Length: 160 Connection: keep-alive Access-Control-Allow-Origin: * Cache-Control: max-age=150 Location: https://vscode.download.prss.microsoft.com/dbazure/download/stable/8b3775030ed1a69b13e4f4c628c612102e30a681/vscode-server-linux-x64.tar.gz Vary: Accept Strict-Transport-Security: max-age=31536000; includeSubDomains request-context: appId=cid-v1:8109e553-6232-492a-afca-b78cea9b431c X-Powered-By: Express X-Content-Type-Options: nosniff X-Source-Commit: a6fe970b22a576c920f8ad19720b231d4844d172 X-SHA256: a0c7d5e39709619882df9b29c62e3d3cd8778b2b54a5d56377f0b72f894b58bd x-azure-ref: 20240201T085701Z-qgd68rdc292dz8m4hy6dxstxf800000002500000000036gz X-Cache: CONFIG_NOCACHE

---response end--- 302 Found Registered socket 4 for persistent reuse. Parsed Strict-Transport-Security max-age = 31536000, includeSubDomains = true Updated HSTS host: update.code.visualstudio.com:443 (max-age: 31536000, includeSubdomains: true) URI content encoding = ‘utf-8’ Location: https://vscode.download.prss.microsoft.com/dbazure/download/stable/8b3775030ed1a69b13e4f4c628c612102e30a681/vscode-server-linux-x64.tar.gz [following] Skipping 160 bytes of body: [Found. Redirecting to https://vscode.download.prss.microsoft.com/dbazure/download/stable/8b3775030ed1a69b13e4f4c628c612102e30a681/vscode-server-linux-x64.tar.gz] done. URI content encoding = None --2024-02-01 08:57:01-- https://vscode.download.prss.microsoft.com/dbazure/download/stable/8b3775030ed1a69b13e4f4c628c612102e30a681/vscode-server-linux-x64.tar.gz Resolving vscode.download.prss.microsoft.com (vscode.download.prss.microsoft.com)... 152.199.21.175, 2606:2800:233:1cb7:261b:1f9c:2074:3c Caching vscode.download.prss.microsoft.com => 152.199.21.175 2606:2800:233:1cb7:261b:1f9c:2074:3c Connecting to vscode.download.prss.microsoft.com (vscode.download.prss.microsoft.com)|152.199.21.175|:443... connected. Created socket 5. Releasing 0x000055b30eedc140 (new refcount 1). Initiating SSL handshake. SSL handshake failed. Closed fd 5 Unable to establish SSL connection. Saving HSTS entries to /home/azureuser/.wget-hsts

shsuman commented 5 months ago

@marijnbaar I can see from the logs that the SSL handshake is failing when trying to access vscode.download.prss.microsoft.com.

It seems like either your proxy or firewall is blocking that URL

marijnbaar commented 5 months ago

Ok, let me rule that out. I will check if it is blocked by our firewall.

shsuman commented 4 months ago

Closing because of inactivity. Please reopen if need be :)