Open dgivens opened 10 years ago
I'm finding that test-kitchen with vagrant-lxc is a better fit.
@fnichol can we increase the number of SSH attempts for asset scp, or make it configurable?
Hmm, wait_for_ssh() should be avoiding this issue.
Isn't there some bit about Ubuntu restarting sshd at startup? Could be why it's connection closed by remote host
I'm also running into this issue, running on Jenkins:
[0m[36m Finished converging <building-tagger-ubuntu-1204> (4m30.00s).
[0m[36m-----> Setting up <building-tagger-ubuntu-1204>...
[0m[36m [SSH] connection failed, retrying (#<Net::SSH::Disconnect: connection closed by remote host>)
[0m[36m [SSH] connection failed, retrying (#<Net::SSH::Disconnect: connection closed by remote host>)
[0m[36m$$$$$$ [SSH] connection failed, terminating (#<Net::SSH::Disconnect: connection closed by remote host>)
After looking into this, it looks like the code that establishes an SSH connection in Test Kitchen retries but doesn't pause between attempts (unlike the #wait_for_ssh
logic). As a result @portertech and I have been testing out https://github.com/test-kitchen/test-kitchen/pull/399 today with reasonable success. I'd like to add a bit more configuration for Driver authors and then merge it into Test Kitchen core.
I too am experiencing this issue, and I believe test-kitchen/test-kitchen/pull/454 may offer a workaround and/or solution if it is merged.
I'm facing same issue, Did someone found a solution?
Is it possible that this issue is not with SSH'ing but with SCP? Can you try to copy something with SCP into the container or install openssh-server+openssh-client (SCP is in client) on the container.
+1
+1
-----> Starting Kitchen (v1.15.0)
-----> Creating <default-centos-73>...
Sending build context to Docker daemon 24.58 MB
Step 1/7 : FROM centos:7
---> 67591570dd29
Step 2/7 : MAINTAINER "msameera" <sameer@mail.com>
---> Using cache
---> dfeeb2440e5a
Step 3/7 : ENV container docker
---> Using cache
---> 3cdba08c07a6
Step 4/7 : EXPOSE 32773
---> Running in 583109b9c836
---> 779a403e4c47
Removing intermediate container 583109b9c836
Step 5/7 : RUN (cd /lib/systemd/system/sysinit.target.wants/; for i in *; do [ $i == systemd-tmpfiles-setup.service ] || rm -f $i; done); rm -f /lib/systemd/system/multi-user.target.wants/*;rm -f /etc/systemd/system/*.wants/*;rm -f /lib/systemd/system/local-fs.target.wants/*; rm -f /lib/systemd/system/sockets.target.wants/*udev*; rm -f /lib/systemd/system/sockets.target.wants/*initctl*; rm -f /lib/systemd/system/basic.target.wants/*;rm -f /lib/systemd/system/anaconda.target.wants/*;
---> Running in 2451fba617e7
---> 2bcb0ff84da1
Removing intermediate container 2451fba617e7
Step 6/7 : VOLUME /sys/fs/cgroup
---> Running in 0166c6ef6be3
---> 509d6f7a7309
Removing intermediate container 0166c6ef6be3
Step 7/7 : CMD /usr/sbin/init
---> Running in fd5db50d5441
---> 2e0bda625ddc
Removing intermediate container fd5db50d5441
Successfully built 2e0bda625ddc
f65e491a3cd55b21cdf56f0bbc308776ae82f0ad9fa7256fcf3d74d2a53e276e
0.0.0.0:32774
Waiting for SSH service on localhost:32774, retrying in 3 seconds
Waiting for SSH service on localhost:32774, retrying in 3 seconds
Waiting for SSH service on localhost:32774, retrying in 3 seconds
Waiting for SSH service on localhost:32774, retrying in 3 seconds
Waiting for SSH service on localhost:32774, retrying in 3 seconds
Waiting for SSH service on localhost:32774, retrying in 3 seconds
Waiting for SSH service on localhost:32774, retrying in 3 seconds
Waiting for SSH service on localhost:32774, retrying in 3 seconds
Waiting for SSH service on localhost:32774, retrying in 3 seconds
Waiting for SSH service on localhost:32774, retrying in 3 seconds
Waiting for SSH service on localhost:32774, retrying in 3 seconds
Waiting for SSH service on localhost:32774, retrying in 3 seconds
Waiting for SSH service on localhost:32774, retrying in 3 seconds
Waiting for SSH service on localhost:32774, retrying in 3 seconds
Waiting for SSH service on localhost:32774, retrying in 3 seconds
Waiting for SSH service on localhost:32774, retrying in 3 seconds
Waiting for SSH service on localhost:32774, retrying in 3 seconds
Waiting for SSH service on localhost:32774, retrying in 3 seconds
Waiting for SSH service on localhost:32774, retrying in 3 seconds
Waiting for SSH service on localhost:32774, retrying in 3 seconds
Waiting for SSH service on localhost:32774, retrying in 3 seconds
Waiting for SSH service on localhost:32774, retrying in 3 seconds
Waiting for SSH service on localhost:32774, retrying in 3 seconds
Waiting for SSH service on localhost:32774, retrying in 3 seconds
Waiting for SSH service on localhost:32774, retrying in 3 seconds
Waiting for SSH service on localhost:32774, retrying in 3 seconds
Waiting for SSH service on localhost:32774, retrying in 3 seconds
Waiting for SSH service on localhost:32774, retrying in 3 seconds
Waiting for SSH service on localhost:32774, retrying in 3 seconds
Waiting for SSH service on localhost:32774, retrying in 3 seconds
Waiting for SSH service on localhost:32774, retrying in 3 seconds
Waiting for SSH service on localhost:32774, retrying in 3 seconds
Waiting for SSH service on localhost:32774, retrying in 3 seconds
Waiting for SSH service on localhost:32774, retrying in 3 seconds
Waiting for SSH service on localhost:32774, retrying in 3 seconds
Waiting for SSH service on localhost:32774, retrying in 3 seconds
Waiting for SSH service on localhost:32774, retrying in 3 seconds
Waiting for SSH service on localhost:32774, retrying in 3 seconds
Waiting for SSH service on localhost:32774, retrying in 3 seconds
Waiting for SSH service on localhost:32774, retrying in 3 seconds
Waiting for SSH service on localhost:32774, retrying in 3 seconds
Waiting for SSH service on localhost:32774, retrying in 3 seconds
Waiting for SSH service on localhost:32774, retrying in 3 seconds
Waiting for SSH service on localhost:32774, retrying in 3 seconds
I'm having the same problem and I found a workaround:
Waiting for SSH service on localhost:32773, retrying in 3 seconds
docker ps
then with the container id: docker exec -it xxxxxx bash
passwd kitchen
I've the same
Waiting for SSH service on localhost:32776, retrying in 3 seconds
Waiting for SSH service on localhost:32776, retrying in 3 seconds
Waiting for SSH service on localhost:32776, retrying in 3 seconds
I got rid of it commenting out run_command: /lib/systemd/systemd
.
Follow up problem is, of course no services are running, can't test my stuff properly :-(
I have the same issue when starting systemd, seems like the vanilla startup-command, which run_command: /bin/systemd
is replacing, is dealing with the ssh server setup
- and is missing otherwise.
Since i require upstart / systemd docker
is road blocked for me in https://github.com/EugenMayer/chef-tinc-cookbook/blob/master/.kitchen.docker.yml
Here's our kitchen.yml, which include our workaround for this issue, the first provision command. The second one fix a testinfra related issue which may be useful for other tools relying on /sbin/init.
---
driver:
name: docker
use_sudo: false
provision_command:
- rm /lib/systemd/system/ssh.service
- '[ ! -f /sbin/init ] && ln -s /lib/systemd/systemd /sbin/init || true'
run_command: /bin/systemd
privileged: true
volume:
- "/sys/fs/cgroup:/sys/fs/cgroup:ro"
dns:
- 1.1.1.1
- 9.9.9.9
transport:
name: sftp
platforms:
- name: stretch
driver_config:
image: jrei/systemd-debian:9
platform: debian
- name: buster
driver_config:
image: jrei/systemd-debian:10
platform: debian
suites:
- name: nitrogen
provisioner:
salt_bootstrap_options: -X -p git -x python2.7 stable 2017.7
- name: fluorine
provisioner:
salt_bootstrap_options: -X -p git -x python2.7 stable 2019.2
provisioner:
name: salt_solo
salt_install: bootstrap
is_file_root: true
require_chef: true
salt_copy_filter:
- .git
dependencies:
- name: common
repo: git
source: https://gitlab+deploy-token-6:REDACTED@gitlab.REDACTED/salt/common-formula
branch: dev
state_top:
base:
"*":
- bender
pillars_from_files:
pillar.sls: test/pillar.sls
pillars:
top.sls:
base:
"*":
- pillar
verifier:
name: shell
remote_exec: false
command: pytest --junitxml=test/${KITCHEN_INSTANCE}_test_report.xml --html=test/${KITCHEN_INSTANCE}_test_report.html --self-contained-html --color=yes --host="docker://root@${KITCHEN_CONTAINER_ID}" "test/integration/"
i fixed it right now by doing
run_command: /bin/systemd
provision_command:
- apt-get install systemd -y
disable_upstart: false
I'm running into an issue where is appears test-kitchen isn't waiting long enough to SSH when I'm using /sbin/init for my run_command. If I wait a couple of second, I can then ssh into the instance. Is this something that can be addressed by kitchen-docker or are there any workarounds I might employ?
I'm trying to solve the problem of runit not starting when the package has been installed. On Debian systems, which I'm testing on, it starts via the inittab and the runit cookbook handles this by issuing a telinit once the package is installed.