Remote-Builder - ssh: connect to host port 22: Connection timed out

kegobass commented 4 years ago

Affected builder image

gcr.io/cloud-builders-community/remote-builder

Expected Behavior

To be able to run through the examples without error

Actual Behavior

Returning error when trying to SSH into instance ssh: connect to host xxxxxxx port 22: Connection timed out

Steps to Reproduce the Problem

Run gcloud builds submit --config=default.yaml .
export PROJECT=$(gcloud info --format='value(config.project)') export PROJECT_NUMBER=$(gcloud projects describe $PROJECT --format 'value(projectNumber)') export CB_SA_EMAIL=$PROJECT_NUMBER@cloudbuild.gserviceaccount.com gcloud services enable cloudbuild.googleapis.com gcloud services enable compute.googleapis.com gcloud projects add-iam-policy-binding $PROJECT --member=serviceAccount:$CB_SA_EMAIL --role='roles/iam.serviceAccountUser' --role='roles/compute.instanceAdmin.v1' --role='roles/iam.serviceAccountActor'
steps:
- name: gcr.io/$PROJECT_ID/remote-builder env:
- COMMAND=ls -la

Additional Info

Added firewall rule to allow 0.0.0.0/0 ingress for SSH.

Complete error:

gcloud compute scp --compress --recurse ./ admin@builder-41e70933-f387-4243-8726-6b4edc3f2d2c:/home/admin/workspace/ --ssh-key-file=builder-key ssh: connect to host 35.193.194.220 port 22: Connection timed out lost connection ERROR: (gcloud.compute.scp) [/usr/bin/scp] exited with return code [1].
cleanup
gcloud compute instances delete builder-41e70933-f387-4243-8726-6b4edc3f2d2c The following instances will be deleted. Any attached disks configured to be auto-deleted will be deleted unless they are attached to any other instances or the --keep-disks flag is given and specifies them for keeping. Deleting a disk is irreversible and any data on the disk will be lost.
- [builder-41e70933-f387-4243-8726-6b4edc3f2d2c] in [us-central1-f]

Do you want to continue (Y/n)? Deleted [https://www.googleapis.com/compute/v1/projects/dev-monolith-275613/zones/us-central1-f/instances/builder-41e70933-f387-4243-8726-6b4edc3f2d2c]. ERROR ERROR: build step 0 "gcr.io/cloud-solutions-images/remote-builder:v0.3.1" failed: step exited with non-zero status: 1

mstallmo commented 4 years ago

:+1: Running into this issue as well with the same permissive ssh configuration

tgadam commented 4 years ago

I also encountered this. It appears to be general flakiness with the Google CloudBuild network. I was able to work around it by adding ssh flags to increase the number of connection attempts. Here are the flags that worked for me: ConnectionAttempts=20 ConnectTimeout=5

This means that ssh and scp (make sure to use --scp-flag for scp) will try to connect, wait 5 seconds, and then try again 19 more times. It's possible you may need to adjust these if the network connectivity gets worse but these seem to work consistently for me now.

Example modified commands in run-builder.sh:

          $${GCLOUD} compute scp --compress --recurse \
              --scp-flag="-o ConnectionAttempts=20" --scp-flag="-o ConnectTimeout=5" \
              $(pwd) $${USERNAME}@$${INSTANCE_NAME}:$${REMOTE_WORKSPACE} \
              --ssh-key-file=$${KEYNAME}

           $${GCLOUD} compute ssh --ssh-key-file=$${KEYNAME} \
              --ssh-flag="-o ConnectionAttempts=20" --ssh-flag="-o ConnectTimeout=5" \
              $${USERNAME}@$${INSTANCE_NAME} -- $${COMMAND}

romiafrizal commented 4 years ago

ssh: connect to host 35.xxx.xxx.xxx port 22: Connection refused

guy-david commented 4 years ago

I added the following to my remote-builder in order to wait and verify the machine is reachable:

CONNECTION_RETRIES=${CONNECTION_RETRIES:-10}

...

retries=0
while ! ${GCLOUD} compute ssh --ssh-key-file=${KEYNAME} \
                  ${USERNAME}@${INSTANCE_NAME} -- exit
do
    retries=$((retries+1))
    if [[ "$retries" -lt $CONNECTION_RETRIES ]]; then
        echo "SSH not ready. Trying again in 5 sec..."
        sleep 5
    else
        echo "ERROR: Couldn't connect to ${INSTANCE_NAME} through SSH"
        exit 1
    fi
done

ezequielsousabr commented 1 year ago

I have the same problem to connect to the remote machine. I added the key publishes and tried to connect with ssh -i id_rsa username@ipexterno and it is not working.

Connection timed out

You can connect only via the Google cloud CLI by clicking ssh connection in instances.

vitaliejicol commented 1 year ago

Running into the same issue while trying to run gcloud compute scp...., any work arounds besides the above ones?

GoogleCloudPlatform / cloud-builders-community