search

adnichols / kitchen-docker-api

Docker driver for test-kitchen using a ruby based docker client
Other
11 stars 2 forks source link

Docker container destroy false fails, leading to kitchen reporting failure when tests pass #4

Open NathanZook opened 9 years ago

NathanZook commented 9 years ago

Macintosh:wp_docker_deploy-cookbook nzook$ boot2docker version Client version: v1.2.0 Git commit: a551732 Macintosh:wp_docker_deploy-cookbook nzook$ docker version Client version: 1.2.0 Client API version: 1.14 Go version (client): go1.3.1 Git commit (client): fa7b24f OS/Arch (client): darwin/amd64 Server version: 1.2.0 Server API version: 1.14 Go version (server): go1.3.1 Git commit (server): fa7b24f

...

   Finished in 0.31451 seconds
   12 examples, 0 failures
   Finished verifying <default-ubuntu-1404> (0m2.22s).
-----> Destroying <default-ubuntu-1404>...
>>>>>> ------Exception------
>>>>>> Class: Kitchen::ActionFailed
>>>>>> Message: Failed to complete #destroy action: [Expected(200..204) <=> Actual(500 InternalServerError)]
>>>>>> ----------------------
>>>>>> Please see .kitchen/logs/kitchen.log for more details
>>>>>> Also try running `kitchen diagnose --all` for configuration

Macintosh:wp_docker_deploy-cookbook nzook$ docker ps -a
CONTAINER ID        IMAGE               COMMAND             CREATED             STATUS              PORTS               NAMES
Macintosh:wp_docker_deploy-cookbook nzook$ docker images
REPOSITORY                             TAG                 IMAGE ID            CREATED             VIRTUAL SIZE
f4f5ca0cc370 6 minutes ago 563.4 MB
adnichols commented 9 years ago

Can you show me the kitchen.yml for this? Or the output of kitchen diagnose? The subject references "destroy false" so I'm wondering if you are trying to tell kitchen to not destroy the container on success & that's triggering a failure? Or am I misunderstanding?

NathanZook commented 9 years ago

It is attempting to destroy on success, the deamon reports a fail, but the container in fact goes away.


---
driver:
  name: docker

provisioner:
  name: chef_zero
  log_level: debug
  require_chef_omnibus: 11.10.0

platforms:
  - name: ubuntu-14.04
    driver_config:
      socket:  <%= ENV['DOCKER_HOST'] %>
      image: 'docker.util.pages/wp_ops/base-ubuntu:trusty'
      platform: ubuntu
      provision_command:
        - apt-get install -y git
        - export DOCKER_HOST=<%= ENV['DOCKER_HOST'] %>
      dns:
        - 172.25.0.20
        - 172.25.0.21
      privileged: true

suites:
  - name: default
    run_list:
      - recipe[test_wrapper::default]
      - recipe[wp_docker_deploy::default]
    attributes:
      wp:
        base:
          upgrade_packages: []
          install_packages: []
        docker:
          cleanup_interval: 0
lfjewett commented 9 years ago

I just thought it might be helpful to add to this report we are also seeing the same problem. The output is listed below. Out of the last 15 instances only 2 have failed and I have not been able to determine a way to reproduce it. The Docker image 'is' being removed successfully even though the error occurs. The image is 'not' gone prior to running the destroy that fails.

       Finished converging <default-centos> (1m36.42s).
-----> Kitchen is finished. (1m36.89s)
+ kitchen destroy
-----> Starting Kitchen (v1.2.2.dev)
-----> Destroying <default-centos>...
>>>>>> ------Exception-------
>>>>>> Class: Kitchen::ActionFailed
>>>>>> Message: Failed to complete #destroy action: [Expected(200..204) <=> Actual(500 InternalServerError)
]
>>>>>> ----------------------
>>>>>> Please see .kitchen/logs/kitchen.log for more details
>>>>>> Also try running `kitchen diagnose --all` for configuration
NathanZook commented 9 years ago

The obvious solution is to rescue the exception, check that the fail message matches the pattern, then sleep a second & retry. Of course, we expect that to fail because the image is already deleted, but we can rescue that.... Bleh.

tknerr commented 9 years ago

Same here, reliably reproducible:

After logging in via ssh I could successfully execute bundle exec kitchen destroy though

tknerr commented 9 years ago

This is likely related to portertech/kitchen-docker#98 (same error behaviour, just different stacktrace)