search

loft-sh / devpod-provider-gcloud

DevPod on Google Cloud
Mozilla Public License 2.0
5 stars 4 forks source link

4047: 'Failed to lookup instance' - IAP Error #27

Open vcolmenar opened 3 months ago

vcolmenar commented 3 months ago

Environment: Windows 11 Devpod: 0.5.19 Gcloud Provider Version: 0.0.11

Issue VM gets created with public IP disabled, but the IAP tunnel fails to create

In my gcloud logs, I see the following attempting to be executed by Devpod (see full log at the very bottom):

`2024-08-23 23:39:18,116 DEBUG    root            Running [gcloud.compute.start-iap-tunnel] with arguments: [--local-host-port: "<googlecloudsdk.calliope.arg_parsers.HostPort object at 0x00000184029AF2D0>", --zone: "https://www.googleapis.com/compute/v1/projects/devpod-test-project/zones/`

Executing this manually via gcloud cli returns the same error

PS C:\Users\devpod_lab> gcloud compute start-iap-tunnel devpod-devpod-shared-gcloud-d871a 22 --zone="https://www.googleapis.com/compute/v1/projects/devpod-test-project/zones/us-central1-c" --project=devpod-test-project
Picking local unused port [57809].
WARNING:

To increase the performance of the tunnel, consider installing NumPy. For instructions,
please see https://cloud.google.com/iap/docs/using-tcp-forwarding#increasing_the_tcp_upload_bandwidth

Testing if tunnel connection works.
ERROR: (gcloud.compute.start-iap-tunnel) While checking if a connection can be made: Error while connecting [4047: 'Failed to lookup instance'].
PS C:\Users\devpod_lab>

I believe it should be executing it like so (notice that --zone value isn't a URI):

PS C:\Users\devpod_lab> gcloud compute start-iap-tunnel devpod-devpod-shared-gcloud-d871a 22 --zone=us-central1-c --project=devpod-test-project
Picking local unused port [59768].
WARNING:

To increase the performance of the tunnel, consider installing NumPy. For instructions,
please see https://cloud.google.com/iap/docs/using-tcp-forwarding#increasing_the_tcp_upload_bandwidth

Testing if tunnel connection works.
Listening on port [59768].

Full gcloud log from failed devpod startup:

2024-08-23 23:39:17,998 DEBUG    root            Loaded Command Group: ['gcloud', 'compute']
2024-08-23 23:39:18,112 DEBUG    root            Loaded Command Group: ['gcloud', 'compute', 'start_iap_tunnel']
2024-08-23 23:39:18,116 DEBUG    root            Running [gcloud.compute.start-iap-tunnel] with arguments: [--local-host-port: "<googlecloudsdk.calliope.arg_parsers.HostPort object at 0x00000184029AF2D0>", --zone: "https://www.googleapis.com/compute/v1/projects/devpod-test-project/zones/us-central1-a", INSTANCE_NAME: "devpod-devpod-shared-gcloud-d871a", INSTANCE_PORT: "22"]
2024-08-23 23:39:18,119 WARNING  root            

To increase the performance of the tunnel, consider installing NumPy. For instructions,
please see https://cloud.google.com/iap/docs/using-tcp-forwarding#increasing_the_tcp_upload_bandwidth

2024-08-23 23:39:18,121 INFO     ___FILE_ONLY___ Testing if tunnel connection works.

2024-08-23 23:39:18,239 DEBUG    root            [-1] user-agent [gcloud/478.0.0 command/gcloud.compute.start-iap-tunnel invocation-id/1b64348c2b14468993fac9c7b6c949e3 environment/None environment-version/None client-os/WINDOWS client-os-ver/10.0.22631 client-pltf-arch/x86_64 interactive/True from-script/False python/3.11.8 term/ (Windows NT 10.0.22631)]
2024-08-23 23:39:18,239 DEBUG    root            credentials type for _GetAccessTokenCallback is [<googlecloudsdk.core.credentials.google_auth_credentials.Credentials object at 0x00000184029AF810>].
2024-08-23 23:39:18,241 DEBUG    root            [-1] Using new websocket library
2024-08-23 23:39:18,243 INFO     root            [-1] Connecting with URL ['wss://tunnel.cloudproxy.app/v4/connect?project=devpod-test-project&port=22&newWebsocket=True&zone=https%3A%2F%2Fwww.googleapis.com%2Fcompute%2Fv1%2Fprojects%2Fdevpod-test-project%2Fzones%2Fus-central1-a&instance=devpod-devpod-shared-gcloud-d871a&interface=nic0']
2024-08-23 23:39:19,017 INFO     root            [-1] Received WebSocket Close message [4047: 'Failed to lookup instance'].
2024-08-23 23:39:19,297 DEBUG    urllib3.connectionpool Starting new HTTPS connection (1): compute.googleapis.com:443
2024-08-23 23:39:20,250 DEBUG    urllib3.connectionpool https://compute.googleapis.com:443 "GET /compute/v1/projects/devpod-test-project/zones/us-central1-a/instances/devpod-devpod-shared-gcloud-d871a?alt=json HTTP/1.1" 200 None
2024-08-23 23:39:20,256 DEBUG    root            (gcloud.compute.start-iap-tunnel) While checking if a connection can be made: Error while connecting [4047: 'Failed to lookup instance'].
Traceback (most recent call last):
  File "C:\Users\devpod_lab\AppData\Local\Google\Cloud SDK\google-cloud-sdk\bin\..\lib\googlecloudsdk\command_lib\compute\iap_tunnel.py", line 819, in Run
    self._TestConnection()
  File "C:\Users\devpod_lab\AppData\Local\Google\Cloud SDK\google-cloud-sdk\bin\..\lib\googlecloudsdk\command_lib\compute\iap_tunnel.py", line 847, in _TestConnection
    conn = self._tunneler._InitiateConnection(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\devpod_lab\AppData\Local\Google\Cloud SDK\google-cloud-sdk\bin\..\lib\googlecloudsdk\command_lib\compute\iap_tunnel.py", line 734, in _InitiateConnection
    new_websocket.InitiateConnection()
  File "C:\Users\devpod_lab\AppData\Local\Google\Cloud SDK\google-cloud-sdk\bin\..\lib\googlecloudsdk\api_lib\compute\iap_tunnel_websocket.py", line 152, in InitiateConnection
    self._WaitForOpenOrRaiseError()
  File "C:\Users\devpod_lab\AppData\Local\Google\Cloud SDK\google-cloud-sdk\bin\..\lib\googlecloudsdk\api_lib\compute\iap_tunnel_websocket.py", line 444, in _WaitForOpenOrRaiseError
    raise ConnectionCreationError(error_msg)
googlecloudsdk.api_lib.compute.iap_tunnel_websocket.ConnectionCreationError: Error while connecting [4047: 'Failed to lookup instance'].

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\Users\devpod_lab\AppData\Local\Google\Cloud SDK\google-cloud-sdk\bin\..\lib\googlecloudsdk\calliope\cli.py", line 998, in Execute
    resources = calliope_command.Run(cli=self, args=args)
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\devpod_lab\AppData\Local\Google\Cloud SDK\google-cloud-sdk\bin\..\lib\googlecloudsdk\calliope\backend.py", line 815, in Run
    resources = command_instance.Run(args)
                ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\devpod_lab\AppData\Local\Google\Cloud SDK\google-cloud-sdk\bin\..\lib\surface\compute\start_iap_tunnel.py", line 155, in Run
    raise e
  File "C:\Users\devpod_lab\AppData\Local\Google\Cloud SDK\google-cloud-sdk\bin\..\lib\surface\compute\start_iap_tunnel.py", line 146, in Run
    iap_tunnel_helper.Run()
  File "C:\Users\devpod_lab\AppData\Local\Google\Cloud SDK\google-cloud-sdk\bin\..\lib\googlecloudsdk\command_lib\compute\iap_tunnel.py", line 821, in Run
    raise iap_tunnel_websocket.ConnectionCreationError(
googlecloudsdk.api_lib.compute.iap_tunnel_websocket.ConnectionCreationError: While checking if a connection can be made: Error while connecting [4047: 'Failed to lookup instance'].
2024-08-23 23:39:20,260 ERROR    root            (gcloud.compute.start-iap-tunnel) While checking if a connection can be made: Error while connecting [4047: 'Failed to lookup instance'].
2024-08-23 23:39:20,260 DEBUG    root            Stopping server.
pascalbreuninger commented 2 months ago

Hey @vcolmenar, thanks for reporting the issue and the detailed logs, we'll look into it