search

sous-chefs / apache2

Development repository for the apache2 cookbook
https://supermarket.chef.io/cookbooks/apache2
Apache License 2.0
285 stars 550 forks source link

Function precedence issue with `libexec_dir` when included with docker cookbook. #605

Closed estenrye closed 5 years ago

estenrye commented 5 years ago

Brief Description

There exists a function precedence issue with these methods: https://github.com/chef-cookbooks/docker/blob/d96b54605b845fc5f34068265569b91a7eb2dee6/libraries/docker_service_base.rb#L95 https://github.com/sous-chefs/apache2/blob/2fcb45ec68d3769b824530f21dfb613390a8e1ce/libraries/helpers.rb#L84

Cookbook version

6.0.0

Chef-client version

14.10.9

Platform Details

CentOS 7.6 Running in VMWare CentOS 7.6 Running in Virtualbox

Scenario

Converge a cookbook that depends on both the docker cookbook and the apache2 cookbook version 5.2.2 or later.

Steps to Reproduce

git clone https://github.com/estenrye/docker_apach2_cookbook_precedence_issue_example.git
kitchen converge
kitchen verify

Expected Result

Cookbook should converge without issue.

Actual Result

esten.rye@USMAP-U102:~/github/issue_example$ kitchen create
-----> Starting Kitchen (v1.23.2)
-----> Creating <default-centos-7>...
       Bringing machine 'default' up with 'virtualbox' provider...
       ==> default: This machine used to live in /home/esten.rye/github/docker_apach2_cookbook_precedence_issue_example/.kitchen/kitchen-vagrant/default-centos-7 but it's now at /home/esten.rye/github/issue_example/.kitchen/kitchen-vagrant/default-centos-7.
       ==> default: Depending on your current provider you may need to change the name of
       ==> default: the machine to run it as a different machine.
       ==> default: Checking if box 'bento/centos-7' is up to date...
       ==> default: Setting the name of the VM: kitchen-issue_example-default-centos-7
       ==> default: Clearing any previously set forwarded ports...
       ==> default: Fixed port collision for 22 => 2222. Now on port 2205.
       ==> default: Vagrant has detected a configuration issue which exposes a
       ==> default: vulnerability with the installed version of VirtualBox. The
       ==> default: current guest is configured to use an E1000 NIC type for a
       ==> default: network adapter which is vulnerable in this version of VirtualBox.
       ==> default: Ensure the guest is trusted to use this configuration or update
       ==> default: the NIC type using one of the methods below:
       ==> default:
       ==> default:   https://www.vagrantup.com/docs/virtualbox/configuration.html#default-nic-type
       ==> default:   https://www.vagrantup.com/docs/virtualbox/networking.html#virtualbox-nic-type
       ==> default: Clearing any previously set network interfaces...
       ==> default: Preparing network interfaces based on configuration...
           default: Adapter 1: nat
       ==> default: Forwarding ports...
           default: 22 (guest) => 2205 (host) (adapter 1)
       ==> default: Booting VM...
       ==> default: Waiting for machine to boot. This may take a few minutes...
           default: SSH address: 127.0.0.1:2205
           default: SSH username: vagrant
           default: SSH auth method: private key
           default:
           default: Vagrant insecure key detected. Vagrant will automatically replace
           default: this with a newly generated keypair for better security.
           default:
           default: Inserting generated public key within guest...
           default: Removing insecure key from the guest if it's present...
           default: Key inserted! Disconnecting and reconnecting using new SSH key...
       ==> default: Machine booted and ready!
       ==> default: Checking for guest additions in VM...
       ==> default: Setting hostname...
       ==> default: Mounting shared folders...
           default: /tmp/omnibus/cache => /home/esten.rye/.kitchen/cache
       ==> default: Machine not provisioned because `--no-provision` is specified.
       [SSH] Established
       Vagrant instance <default-centos-7> created.
       Finished creating <default-centos-7> (0m48.77s).
-----> Kitchen is finished. (0m52.04s)
esten.rye@USMAP-U102:~/github/issue_example$ kitchen converge
-----> Starting Kitchen (v1.23.2)
-----> Converging <default-centos-7>...
       Preparing files for transfer
       Preparing dna.json
       Resolving cookbook dependencies with Berkshelf 7.0.6...
       Removing non-cookbook files before transfer
       Preparing validation.pem
       Preparing client.rb
       el 7 x86_64
       Getting information for chef stable 14 for el...
       downloading https://omnitruck.chef.io/stable/chef/metadata?v=14&p=el&pv=7&m=x86_64
         to file /tmp/install.sh.2896/metadata.txt
       trying wget...
       sha1     7fa8289867cd8e91d1e325099a71c3e04e8d7f99
       sha256   6149bbd478318f5bea029274860ffeb9686d04d6534b77cc1ff48043b6a3a2a7
       url      https://packages.chef.io/files/stable/chef/14.10.9/el/7/chef-14.10.9-1.el7.x86_64.rpm
       version  14.10.9
       downloaded metadata file looks valid...
       /tmp/omnibus/cache/chef-14.10.9-1.el7.x86_64.rpm exists
       Comparing checksum with sha256sum...
       Installing chef 14
       installing with rpm...
       warning: /tmp/omnibus/cache/chef-14.10.9-1.el7.x86_64.rpm: Header V4 DSA/SHA1 Signature, key ID 83ef826a: NOKEY
       Preparing...                          ################################# [100%]
       Updating / installing...
          1:chef-14.10.9-1.el7               ################################# [100%]
       Thank you for installing Chef!
       Transferring files to <default-centos-7>
       Starting Chef Client, version 14.10.9
       Creating a new client identity for default-centos-7 using the validator key.
       resolving cookbooks for run list: ["docker_apache2_precedence_issue::default"]
       Synchronizing Cookbooks:
         - docker_apache2_precedence_issue (1.0.0)
         - docker (4.9.2)
         - virtualbox-install (2.0.0)
         - dmg (4.1.1)
         - apt (7.1.1)
         - windows (5.2.3)
         - apache2 (6.0.0)
       Installing Cookbook Gems:
       Compiling Cookbooks...
       Converging 6 resources
       Recipe: docker_apache2_precedence_issue::default
         * yum_package[yum-utils, device-mapper-persistent-data, lvm2] action install (up to date)
         * docker_installation_package[default] action create
           * yum_repository[Docker] action create
             * template[/etc/yum.repos.d/Docker.repo] action create
        - create new file /etc/yum.repos.d/Docker.repo
        - update content in file /etc/yum.repos.d/Docker.repo from none to 6d9a6c
        --- /etc/yum.repos.d/Docker.repo        2019-02-26 21:05:09.942732102 +0000
        +++ /etc/yum.repos.d/.chef-Docker20190226-3017-naoh23.repo      2019-02-26 21:05:09.942732102 +0000
        @@ -1 +1,11 @@
        +# This file was generated by Chef
        +# Do NOT modify this file by hand.
        +
        +[Docker]
        +name=Docker Stable repository
        +baseurl=https://download.docker.com/linux/centos/7/x86_64/stable
        +enabled=1
        +fastestmirror_enabled=0
        +gpgcheck=1
        +gpgkey=https://download.docker.com/linux/centos/gpg
        - change mode from '' to '0644'
        - restore selinux security context
             * execute[yum clean metadata Docker] action run
        - execute yum clean metadata --disablerepo=* --enablerepo=Docker
             * execute[yum-makecache-Docker] action run
        - execute yum -q -y makecache --disablerepo=* --enablerepo=Docker
             * ruby_block[package-cache-reload-Docker] action create
        - execute the ruby block package-cache-reload-Docker
             * execute[yum clean metadata Docker] action nothing (skipped due to action :nothing)
             * execute[yum-makecache-Docker] action nothing (skipped due to action :nothing)
             * ruby_block[package-cache-reload-Docker] action nothing (skipped due to action :nothing)

           * yum_package[docker-ce] action install
             - install version 18.06.0.ce-3.el7 of package docker-ce

         * docker_service_manager_systemd[default] action start
           * directory[/usr/libexec/docker] action create
             - create new directory /usr/libexec/docker
             - change mode from '' to '0755'
             - change owner from '' to 'root'
             - change group from '' to 'root'
             - restore selinux security context
           * template[/usr/libexec/docker/docker-wait-ready] action create
             - create new file /usr/libexec/docker/docker-wait-ready
             - update content in file /usr/libexec/docker/docker-wait-ready from none to 64d78d
             --- /usr/libexec/docker/docker-wait-ready  2019-02-26 21:06:48.063153733 +0000
             +++ /usr/libexec/docker/.chef-docker-wait-ready20190226-3017-1jexzje       2019-02-26 21:06:48.052153381 +0000
             @@ -1 +1,11 @@
             +#!/usr/bin/env bash
             +i=0
             +while [ $i -lt 40 ]; do
             +  /usr/bin/docker  ps | head -n 1 | grep ^CONTAINER > /dev/null 2>&1
             +  [ $? -eq 0 ] && break
             +  ((i++))
             +  sleep 0.5
             +done
             +[ $i -eq 40 ] && exit 1
             +exit 0
             - change mode from '' to '0755'
             - change owner from '' to 'root'
             - change group from '' to 'root'
             - restore selinux security context
           * template[/lib/systemd/system/docker.socket] action create
             - create new file /lib/systemd/system/docker.socket
             - update content in file /lib/systemd/system/docker.socket from none to 7d06f6
             --- /lib/systemd/system/docker.socket      2019-02-26 21:06:48.122155625 +0000
             +++ /lib/systemd/system/.chef-docker20190226-3017-1my1crx.socket   2019-02-26 21:06:48.122155625 +0000
             @@ -1 +1,13 @@
             +[Unit]
             +Description=Docker Socket for the API
             +PartOf=docker.service
             +
             +[Socket]
             +ListenStream=/var/run/docker.sock
             +SocketMode=0660
             +SocketUser=root
             +SocketGroup=docker
             +
             +[Install]
             +WantedBy=sockets.target
             - change mode from '' to '0644'
             - change owner from '' to 'root'
             - change group from '' to 'root'
             - restore selinux security context
           * template[/lib/systemd/system/docker.service] action create
             - update content in file /lib/systemd/system/docker.service from f67888 to 1eff05
             --- /lib/systemd/system/docker.service     2018-07-18 19:01:19.000000000 +0000
             +++ /lib/systemd/system/.chef-docker20190226-3017-163uang.service  2019-02-26 21:06:48.178157421 +0000
             @@ -1,7 +1,8 @@
       [Unit]
       Description=Docker Application Container Engine
       Documentation=https://docs.docker.com
             -After=network-online.target firewalld.service
             +After=network-online.target docker.socket firewalld.service
             +Requires=docker.socket
       Wants=network-online.target

       [Service]
             @@ -9,16 +10,16 @@
       # the default is not to use systemd for cgroups because the delegate issues still
       # exists and systemd currently does not support the cgroup feature set required
       # for containers run by docker
             -ExecStart=/usr/bin/dockerd
             +ExecStart=/usr/bin/dockerd -H fd://
       ExecReload=/bin/kill -s HUP $MAINPID
             +LimitNOFILE=1048576
       # Having non-zero Limit*s causes performance problems due to accounting overhead
       # in the kernel. We recommend using cgroups to do container-local accounting.
             -LimitNOFILE=infinity
       LimitNPROC=infinity
       LimitCORE=infinity
       # Uncomment TasksMax if your systemd version supports it.
       # Only systemd 226 and above support this version.
             -#TasksMax=infinity
             +TasksMax=infinity
       TimeoutStartSec=0
       # set delegate yes so that systemd does not reset the cgroups of docker containers
       Delegate=yes
             - restore selinux security context
           * template[/etc/systemd/system/docker.socket] action create
             - create new file /etc/systemd/system/docker.socket
             - update content in file /etc/systemd/system/docker.socket from none to a64128
             --- /etc/systemd/system/docker.socket      2019-02-26 21:06:48.222158832 +0000
             +++ /etc/systemd/system/.chef-docker20190226-3017-17zp5y3.socket   2019-02-26 21:06:48.222158832 +0000
             @@ -1 +1,12 @@
             +[Unit]
             +Description=Docker Socket for the API
             +PartOf=docker.service
             +
             +[Socket]
             +ListenStream=/var/run/docker.sock
             +SocketGroup=docker
             +
             +
             +[Install]
             +WantedBy=sockets.target
             - change mode from '' to '0644'
             - change owner from '' to 'root'
             - change group from '' to 'root'
             - restore selinux security context
           * template[/etc/systemd/system/docker.service] action create
             - create new file /etc/systemd/system/docker.service
             - update content in file /etc/systemd/system/docker.service from none to 3ed9f0
             --- /etc/systemd/system/docker.service     2019-02-26 21:06:48.278160628 +0000
             +++ /etc/systemd/system/.chef-docker20190226-3017-h9tjtp.service   2019-02-26 21:06:48.278160628 +0000
             @@ -1 +1,29 @@
             +[Unit]
             +Description=Docker Application Container Engine
             +Documentation=https://docs.docker.com
             +After=network-online.target docker.socket firewalld.service
             +Requires=docker.socket
             +Wants=network-online.target
             +
             +[Service]
             +Type=notify
             +ExecStartPre=/sbin/sysctl -w net.ipv4.ip_forward=1
             +ExecStartPre=/sbin/sysctl -w net.ipv6.conf.all.forwarding=1
             +ExecStart=/usr/bin/dockerd  --group=docker --log-driver=json-file --pidfile=/var/run/docker.pid
             +ExecStartPost=/usr/lib64/httpd/modules/docker-wait-ready
             +ExecReload=/bin/kill -s HUP $MAINPID
             +LimitNOFILE=1048576
             +LimitNPROC=infinity
             +LimitCORE=infinity
             +TasksMax=infinity
             +TimeoutStartSec=0
             +Delegate=yes
             +KillMode=process
             +Restart=always
             +StartLimitBurst=3
             +StartLimitInterval=60s
             +
             +
             +[Install]
             +WantedBy=multi-user.target
             - change mode from '' to '0644'
             - change owner from '' to 'root'
             - change group from '' to 'root'
             - restore selinux security context
           * execute[systemctl daemon-reload] action run
             - execute /bin/systemctl daemon-reload
           * execute[systemctl try-restart docker] action run
             - execute /bin/systemctl try-restart docker
           * execute[systemctl daemon-reload] action nothing (skipped due to action :nothing)
           * execute[systemctl try-restart docker] action nothing (skipped due to action :nothing)
           * service[docker] action enable
             - enable service service[docker]
           * service[docker] action start

             ================================================================================
             Error executing action `start` on resource 'service[docker]'
             ================================================================================

             Mixlib::ShellOut::ShellCommandFailed
             ------------------------------------
             Expected process to exit with [0], but received '1'
             ---- Begin output of /bin/systemctl --system start docker ----
             STDOUT:
             STDERR: Job for docker.service failed because the control process exited with error code. See "systemctl status docker.service" and "journalctl -xe" for details.
             ---- End output of /bin/systemctl --system start docker ----
             Ran /bin/systemctl --system start docker returned 1

             Resource Declaration:
             ---------------------
             # In /tmp/kitchen/cache/cookbooks/docker/libraries/docker_service_manager_systemd.rb

       92:       service docker_name do
       93:         provider Chef::Provider::Service::Systemd
       94:         supports status: true
       95:         action [:enable, :start]
       96:         only_if { ::File.exist?("/lib/systemd/system/#{docker_name}.service") }
       97:         retries 1
       98:       end
       99:     end

             Compiled Resource:
             ------------------
             # Declared in /tmp/kitchen/cache/cookbooks/docker/libraries/docker_service_manager_systemd.rb:92:in `block in <class:DockerServiceManagerSystemd>'

             service("docker") do
        provider Chef::Provider::Service::Systemd
        action [:enable, :start]
        updated true
        default_guard_interpreter :default
        service_name "docker"
        enabled true
        running false
        masked false
        pattern "docker"
        declared_type :service
        cookbook_name "docker_apache2_precedence_issue"
        supports {:status=>true}
        retries 1
        only_if { #code block }
             end

             System Info:
             ------------
             chef_version=14.10.9
             platform=centos
             platform_version=7.5.1804
             ruby=ruby 2.5.3p105 (2018-10-18 revision 65156) [x86_64-linux]
             program_name=/opt/chef/bin/chef-client
             executable=/opt/chef/bin/chef-client

           ================================================================================
           Error executing action `start` on resource 'docker_service_manager_systemd[default]'
           ================================================================================

           Mixlib::ShellOut::ShellCommandFailed
           ------------------------------------
           service[docker] (/tmp/kitchen/cache/cookbooks/docker/libraries/docker_service_manager_systemd.rb line 92) had an error: Mixlib::ShellOut::ShellCommandFailed: Expected process to exit with [0], but received '1'
           ---- Begin output of /bin/systemctl --system start docker ----
           STDOUT:
           STDERR: Job for docker.service failed because the control process exited with error code. See "systemctl status docker.service" and "journalctl -xe" for details.
           ---- End output of /bin/systemctl --system start docker ----
           Ran /bin/systemctl --system start docker returned 1

           Resource Declaration:
           ---------------------
           # In /tmp/kitchen/cache/cookbooks/docker_apache2_precedence_issue/recipes/default.rb

             9: docker_service_manager 'default' do
            10:   log_driver 'json-file'
            11:   action :start
            12: end
            13:

           Compiled Resource:
           ------------------
           # Declared in /tmp/kitchen/cache/cookbooks/docker_apache2_precedence_issue/recipes/default.rb:9:in `from_file'

           docker_service_manager_systemd("default") do
             action [:start]
             updated true
             updated_by_last_action true
             default_guard_interpreter :default
             declared_type :docker_service_manager
             cookbook_name "docker_apache2_precedence_issue"
             recipe_name "default"
             log_driver "json-file"
             pidfile "/var/run/docker.pid"
           end

           System Info:
           ------------
           chef_version=14.10.9
           platform=centos
           platform_version=7.5.1804
           ruby=ruby 2.5.3p105 (2018-10-18 revision 65156) [x86_64-linux]
           program_name=/opt/chef/bin/chef-client
           executable=/opt/chef/bin/chef-client

       Running handlers:
       [2019-02-26T21:06:54+00:00] ERROR: Running exception handlers
       Running handlers complete
       [2019-02-26T21:06:54+00:00] ERROR: Exception handlers complete
       Chef Client failed. 16 resources updated in 01 minutes 59 seconds
       [2019-02-26T21:06:54+00:00] FATAL: Stacktrace dumped to /tmp/kitchen/cache/chef-stacktrace.out
       [2019-02-26T21:06:54+00:00] FATAL: Please provide the contents of the stacktrace.out file if you file a bug report
       [2019-02-26T21:06:54+00:00] FATAL: Mixlib::ShellOut::ShellCommandFailed: docker_service_manager_systemd[default] (docker_apache2_precedence_issue::default line 9) had an error: Mixlib::ShellOut::ShellCommandFailed: service[docker] (/tmp/kitchen/cache/cookbooks/doc
ker/libraries/docker_service_manager_systemd.rb line 92) had an error: Mixlib::ShellOut::ShellCommandFailed: Expected process to exit with [0], but received '1'
       ---- Begin output of /bin/systemctl --system start docker ----
       STDOUT:
       STDERR: Job for docker.service failed because the control process exited with error code. See "systemctl status docker.service" and "journalctl -xe" for details.
       ---- End output of /bin/systemctl --system start docker ----
       Ran /bin/systemctl --system start docker returned 1
>>>>>> ------Exception-------
>>>>>> Class: Kitchen::ActionFailed
>>>>>> Message: 1 actions failed.
>>>>>>     Converge failed on instance <default-centos-7>.  Please see .kitchen/logs/default-centos-7.log for more details
>>>>>> ----------------------
>>>>>> Please see .kitchen/logs/kitchen.log for more details
>>>>>> Also try running `kitchen diagnose --all` for configuration
estenrye commented 5 years ago

Related Issue: https://github.com/chef-cookbooks/docker/issues/1066

damacus commented 5 years ago

Duplicated by #609 Fixed by #611

teknofire commented 4 years ago

@damacus I don't believe #611 fixes this issue, it looks like there are two locations that still end up including the helpers into the global scope.

https://github.com/sous-chefs/apache2/blob/master/resources/default_site.rb#L1 https://github.com/sous-chefs/apache2/blob/master/resources/mod_pagespeed.rb#L1

As well as into the action_class block at the end of the files.

ramereth commented 4 years ago

@teknofire can you see if #678 fixes this now?