No such file or directory - /sbin/status

[jarnohenneman]

No such file or directory - /sbin/status - Unable to start or restart the Sensu service on Ubuntu 16.04

Expected Behavior

The correct Ubuntu provider should be selected and service installed.

Current Behavior

Currently it will fail on the installation of Sensu::default or other Sensu recipes.

Possible Solution

-

Steps to Reproduce (for bugs)

1. Installing redis, after that trying to install Sensu services.

   include_recipe 'sensu::server_service'
   include_recipe 'sensu::api_service'
   include_recipe 'sensu::client_service'

2. When running kitchen converge --color --no-log-overwrite --log-level info default-ubuntu-1604 it fails with below error.

Recipe: sensu::default
  * ruby_block[sensu_service_trigger] action create
    - execute the ruby block sensu_service_trigger
  * service[sensu-server] action restart

    ================================================================================
    Error executing action `restart` on resource 'service[sensu-server]'
    ================================================================================

    Errno::ENOENT
    -------------
    No such file or directory - /sbin/status

    Resource Declaration:
    ---------------------
    # In /opt/kitchen/cache/cookbooks/sensu/providers/service.rb

      7:   @sensu_svc ||= service new_resource.service do
      8:     supports :status => true, :restart => true
      9:     retries 3
     10:     retry_delay 5
     11:     action :nothing
     12:     subscribes :restart, resources("ruby_block[sensu_service_trigger]"), :delayed
     13:   end
     14: end

    Compiled Resource:
    ------------------
    # Declared in /opt/kitchen/cache/cookbooks/sensu/providers/service.rb:7:in `load_current_resource'

    service("sensu-server") do
      action [:nothing]
      default_guard_interpreter :default
      service_name "sensu-server"
      enabled nil
      running nil
      masked nil
      pattern "sensu-server"
      declared_type :service
      cookbook_name "sensu"
      retries 3
      retry_delay 5
      supports {:status=>true, :restart=>true}
    end

    System Info:
    ------------
    chef_version=14.3.37
    platform=ubuntu
    platform_version=16.04
    ruby=ruby 2.5.1p57 (2018-03-29 revision 63029) [x86_64-linux]
    program_name=/opt/chef/embedded/bin/chef-client
    executable=/opt/chef/embedded/bin/chef-client

Running handlers:
[2018-07-17T13:47:36+00:00] ERROR: Running exception handlers
Running handlers complete
[2018-07-17T13:47:36+00:00] ERROR: Exception handlers complete
Chef Client failed. 211 resources updated in 05 minutes 56 seconds
[2018-07-17T13:47:36+00:00] FATAL: Stacktrace dumped to /opt/kitchen/cache/chef-stacktrace.out
[2018-07-17T13:47:36+00:00] FATAL: Please provide the contents of the stacktrace.out file if you file a bug report
[2018-07-17T13:47:36+00:00] FATAL: Chef::Exceptions::MultipleFailures: Multiple failures occurred:
* Errno::ENOENT occurred in chef run: sensu_service[sensu-server] (sensu::server_service line 20) had an error: Errno::ENOENT: service[sensu-server] (/opt/kitchen/cache/cookbooks/sensu/providers/service.rb line 7) had an error: Errno::ENOENT: No such file or directory - /sbin/status
* Errno::ENOENT occurred in delayed notification: service[sensu-server] (/opt/kitchen/cache/cookbooks/sensu/providers/service.rb line 7) had an error: Errno::ENOENT: No such file or directory - /sbin/status

Your Environment

• Version of this cookbook used: 5.1.3
• Version of Sensu used: 1.4.2-3
• Version of Chef used: 14.3
• Operating System and version (e.g. CentOS 7, Ubuntu 14.04): dokken\ubuntu-16.04 https://hub.docker.com/r/dokken/ubuntu-16.04/

[majormoses]

Interesting as it should be using systemd on Ubuntu 16.04 and sensu 1.4.2 and I know this worked previously at my last company as that is what they used. I don't recall doing anything special to make this happen.

[majormoses]

Can you try replicating this with older versions of chef?

[majormoses]

I also see you are using dokken can you please include a gist of what your config looks like? I recall needing to specify pid_one_command: /bin/systemd in your .kitchen.yml

[majormoses]

I conferred with a colleague of mine at the last place and we did have to do some magic to make systemd work with sensu in a container. Here is the platform section he gave me:

platforms:
 - name: ubuntu-1604
   named_run_list: debian
   driver:
     image: ubuntu:16.04
     pid_one_command: /bin/systemd
     volumes:
       - /sys/fs/cgroup:/sys/fs/cgroup:ro # required by systemd
     intermediate_instructions: # make systemd work as process 1
       # install addtional packages that can't be installed by cookbook
       - ENV container docker
       - RUN rm -rf /var/lib/apt/lists/*
       - RUN (cd /lib/systemd/system/sysinit.target.wants/; for i in *; do [ $i == systemd-tmpfiles-setup.service ] || rm -f $i; done)
       - RUN rm -f /lib/systemd/system/multi-user.target.wants/*
       - RUN rm -f /etc/systemd/system/*.wants/*
       - RUN rm -f /lib/systemd/system/local-fs.target.wants/*
       - RUN rm -f /lib/systemd/system/sockets.target.wants/*udev*
       - RUN rm -f /lib/systemd/system/sockets.target.wants/*initctl*
       - RUN rm -f /lib/systemd/system/basic.target.wants/*
       - RUN rm -f /lib/systemd/system/anaconda.target.wants/*
       - RUN apt-get -y update
       - RUN apt-get install -y apt-transport-https build-essential zlib1g-dev python-pip sudo net-tools
       - RUN pip install PyYAML && pip install awscli

[jarnohenneman]

Thank you @majormoses , will give it a test to see if its working!

Unfortunately no smoke light yet,,, this is the kitchen Im trying with:

---
driver:
  name: dokken
  chef_version: latest
  privileged: true # because Docker and SystemD/Upstart
  customize:
    memory: '512'

transport:
  name: dokken

provisioner:
  name: dokken

verifier:
  name: inspec

platforms:
  - name: ubuntu-16.04
    driver:
      # image: ubuntu:16.04
      image: dokken/ubuntu-16.04
      pid_one_command: /bin/systemd
      volumes:
        - /sys/fs/cgroup:/sys/fs/cgroup:ro # required by systemd
      intermediate_instructions: # make systemd work as process 1
        # install addtional packages that can't be installed by cookbook
        - ENV container docker
        - RUN rm -rf /var/lib/apt/lists/*
        - RUN (cd /lib/systemd/system/sysinit.target.wants/; for i in *; do [ $i == systemd-tmpfiles-setup.service ] || rm -f $i; done)
        - RUN rm -f /lib/systemd/system/multi-user.target.wants/*
        - RUN rm -f /etc/systemd/system/*.wants/*
        - RUN rm -f /lib/systemd/system/local-fs.target.wants/*
        - RUN rm -f /lib/systemd/system/sockets.target.wants/*udev*
        - RUN rm -f /lib/systemd/system/sockets.target.wants/*initctl*
        - RUN rm -f /lib/systemd/system/basic.target.wants/*
        - RUN rm -f /lib/systemd/system/anaconda.target.wants/*
        - RUN apt-get -y update
        - RUN apt-get install -y apt-transport-https build-essential zlib1g-dev python-pip sudo net-tools
        - RUN pip install PyYAML && pip install awscli
    run_list:
      - recipe[apt]
    attributes:
      apt:
        compile_time_update: true

suites:
- name: default
  run_list:
    - recipe[monitor::master]

[jarnohenneman]

Can you try replicating this with older versions of chef?

It used to work with chef-lxc and chef12, but would like to throw lxc out of the window and go for a container like setup.

[majormoses]

@jarnohenneman I see you are using the monitor cookbook which is quite old and not been updated. If you are using all the default attributes (not properly overriding them) the version of sensu defined does not include a systemd unit file which I believe started shipping in 0.27. I know that when I was there we were also on chef 12 and I am not sure if they have upgraded to chef 13 yet. I can try asking if they have done that yet. That being said at my current org we are using chef 13 our setup but are still sadly on Ubuntu 14.04 so I cant really confirm what other changes would be necessary.

[autarchprinceps]

We are not using the supermarket monitor cookbook. As mentioned we are deploying sensu in version 1.4 not 0.27. It works for us with Chef 12 on VMs / Cloud instances, be it as a test with kitchen or in production, and as mentioned in our usual quick testing environment of kitchen-lxc. Sensu is also entirely installed by the sensu cookbook (meaning the one this discussion is in). The purpose of our cookbook, is to setup the basic checks of every system and the sensu gems/plugins, handlers, etc. necessary to do so, as well as the transport and therefore the target of sensu-clients to which to report to. What we certainly do not modify is the fact that there is a service, or that it is started, enabled, etc. and how that works one different distributions or operating systems.

[majormoses]

It used to work with chef-lxc and chef12, but would like to throw lxc out of the window and go for a container like setup.

Can we try changing one thing at a time to see if its more of a dokken/docker issue and use chef 12?

@autarchprinceps makes sense, I would have no way of knowing that its an internal cookbook based on the code block I was given. I will try to see if I can put together a minimal reproduction case when I have some time. It might be a bit as I have very limited personal time for the next week or so due to some family events.

[majormoses]

To help me reproduce does this happen only on the server component or does it fail on client as well?

[majormoses]

I was not able to reproduce this with a minimal setup, perhaps with more information as to what you are doing in your custom cookbook I will be able to further assist.

Minimal setup: https://github.com/sensu/sensu-chef/compare/feature/local-dokken-testing?expand=1

Output: https://gist.github.com/majormoses/ef35735eaf05b38bda9aba9fd7f96b2a

[jarnohenneman]

Thank you @majormoses , think I have it running let me do some more testing and get back to you

[jarnohenneman]

For me its working now with the following setup;

---
driver:
  name: dokken
  chef_version: latest
  privileged: true # because Docker and SystemD/Upstart

platforms:
  - name: ubuntu-16.04
    driver:
      image: dokken/ubuntu-16.04
      pid_one_command: /bin/systemd

My problem was that I ended up commenting to much recipes, or saying it differently sensu::default is required to run sensu::server_service - sensu::api_service - sensu::client_service, not sure why but above worked for me. Thank you so much @majormoses !

[jarnohenneman]

Its working perfectly @majormoses ! Thank you for your help I'm closing the issue.