search

SumoLogic / sumologic-collector-chef-cookbook

Chef Cookbook for installing and configuring the SumoLogic collector for the Sumo Logic service
Other
30 stars 90 forks source link

Kitchen converge fails #132

Closed majormoses closed 7 years ago

majormoses commented 7 years ago

Using master kitchen converge fails with a compile error, however this works for me when I wrap it in my own cookbook. I assume this needs something like databag or attribute updates in them.

Example failure:

 [2017-06-12T20:50:49+00:00] WARN: ***************** Creating a fake Data Bag with real Sumo key
       [2017-06-12T20:50:49+00:00] WARN: ***************** Creating a fake Data Bag with real Sumo key

       ================================================================================
       Recipe Compile Error in /tmp/kitchen/cache/cookbooks/data-bag-faker/recipes/default.rb
       ================================================================================

       TypeError
       ---------
       no implicit conversion of nil into String

       Cookbook Trace:
       ---------------
         /tmp/kitchen/cache/cookbooks/data-bag-faker/recipes/default.rb:16:in `from_file'

       Relevant File Content:
       ----------------------
       /tmp/kitchen/cache/cookbooks/data-bag-faker/recipes/default.rb:

         9:  
        10:  BAG_PATH = Chef::Config[:data_bag_path]
        11:  TARGET_BAG = 'sumo-creds/api-creds.json'
        12:  BAG_FILE = BAG_PATH + '/' + TARGET_BAG
        13:  BACKUP_FILE = BAG_FILE + '.old'
        14:  
        15:  edit_file = Chef::Util::FileEdit.new(BAG_FILE)
        16>> edit_file.search_file_replace(sumo_access_id_var, sumo_access_id)
        17:  edit_file.search_file_replace(sumo_access_key_var, sumo_access_key)
        18:  edit_file.write_file
        19:  
        20:  File.delete(BACKUP_FILE) if File.exist?(BACKUP_FILE) # FileEdit creates, but does not remove, a backup file.
        21:  

       System Info:
       ------------
       chef_version=13.1.31
       platform=ubuntu
       platform_version=14.04
       ruby=ruby 2.4.1p111 (2017-03-22 revision 58053) [x86_64-linux]
       program_name=chef-client worker: ppid=1256;start=20:50:48;
       executable=/opt/chef/bin/chef-client

       Running handlers:
       [2017-06-12T20:50:49+00:00] ERROR: Running exception handlers
       [2017-06-12T20:50:49+00:00] ERROR: Running exception handlers
       Running handlers complete
       [2017-06-12T20:50:49+00:00] ERROR: Exception handlers complete
       [2017-06-12T20:50:49+00:00] ERROR: Exception handlers complete
       Chef Client failed. 0 resources updated in 01 seconds
       [2017-06-12T20:50:49+00:00] FATAL: Stacktrace dumped to /tmp/kitchen/cache/chef-stacktrace.out
       [2017-06-12T20:50:49+00:00] FATAL: Stacktrace dumped to /tmp/kitchen/cache/chef-stacktrace.out
       [2017-06-12T20:50:49+00:00] FATAL: Please provide the contents of the stacktrace.out file if you file a bug report
       [2017-06-12T20:50:49+00:00] FATAL: Please provide the contents of the stacktrace.out file if you file a bug report
       [2017-06-12T20:50:49+00:00] ERROR: no implicit conversion of nil into String
       [2017-06-12T20:50:49+00:00] ERROR: no implicit conversion of nil into String
       [2017-06-12T20:50:49+00:00] FATAL: Chef::Exceptions::ChildConvergeError: Chef run process exited unsuccessfully (exit code 1)
       [2017-06-12T20:50:49+00:00] FATAL: Chef::Exceptions::ChildConvergeError: Chef run process exited unsuccessfully (exit code 1)

wrapper example:

$ KITCHEN_YAML=.kitchen.docker.yml bundle exec kitchen converge
-----> Starting Kitchen (v1.10.2)
-----> Creating <default-ubuntu-1404>...
       Sending build context to Docker daemon  154.7MB
       Step 1/17 : FROM ubuntu:14.04
        ---> b1719e1db756
       Step 2/17 : RUN dpkg-divert --local --rename --add /sbin/initctl
        ---> Using cache
        ---> 49b1617489c7
       Step 3/17 : RUN ln -sf /bin/true /sbin/initctl
        ---> Using cache
        ---> 15a6417c4efa
       Step 4/17 : ENV DEBIAN_FRONTEND noninteractive
        ---> Using cache
        ---> bce335d0d051
       Step 5/17 : ENV container docker
        ---> Using cache
        ---> e655ffb132ea
       Step 6/17 : RUN apt-get update
        ---> Using cache
        ---> bb0a0bb089e5
       Step 7/17 : RUN apt-get install -y sudo openssh-server curl lsb-release
        ---> Using cache
        ---> c7685847dfef
       Step 8/17 : RUN if ! getent passwd kitchen; then                 useradd -d /home/kitchen -m -s /bin/bash -p '*' kitchen;               fi
        ---> Using cache
        ---> afbe99fe0450
       Step 9/17 : RUN echo "kitchen ALL=(ALL) NOPASSWD: ALL" >> /etc/sudoers
        ---> Using cache
        ---> 0604e3adead1
       Step 10/17 : RUN echo "Defaults !requiretty" >> /etc/sudoers
        ---> Using cache
        ---> 2f237a24896e
       Step 11/17 : RUN mkdir -p /home/kitchen/.ssh
        ---> Using cache
        ---> bceae3a56bc7
       Step 12/17 : RUN chown -R kitchen /home/kitchen/.ssh
        ---> Using cache
        ---> 2410df223748
       Step 13/17 : RUN chmod 0700 /home/kitchen/.ssh
        ---> Using cache
        ---> 78be5c052a0a
       Step 14/17 : RUN touch /home/kitchen/.ssh/authorized_keys
        ---> Using cache
        ---> 607889476201
       Step 15/17 : RUN chown kitchen /home/kitchen/.ssh/authorized_keys
        ---> Using cache
        ---> a19656ea296e
       Step 16/17 : RUN chmod 0600 /home/kitchen/.ssh/authorized_keys
        ---> Using cache
        ---> c914b33e7fd8
       Step 17/17 : RUN echo ssh-rsa\ AAAAB3NzaC1yc2EAAAADAQABAAABAQDVdI8OBdXaZNT01HCUtXxwhkrZ3ILY90jHdSrNCzvxecmRihUm4Kv1/AMmbhqpXpMqQNyKMNAB1tTnXn2oBL4GpEfE\+F\+g1T041vRX3yqOHYHB9wJaeosa3Nu5WaYm0pr6O6sCg/VXrzMuumrD4/HV45Q9nWUr5PMf4alw3XP5Rf1YawvdAdJm\+abEC7EWBfRk9xaQdRX7GbScrCaOhEDtQj\+\+\+qutYHYg6n8li8rCTxVfi0GRMp5CkdiPJ\+PC9l7SFRKIzGkvekWawWeU/plyjqsaA/IKLUEox49GjB1hKms3BA1T9VCzk\+agnO2fGNKth\+Fg6tYsraXkzAFKoCID\ kitchen_docker_key >> /home/kitchen/.ssh/authorized_keys
        ---> Using cache
        ---> c53ed92d95bf
       Successfully built c53ed92d95bf
       87e2d7a9159a720f5e6b9af72cda19397db34f98ccf6af18b0b65bc8047c3a22
       0.0.0.0:32768
       [SSH] Established
       Finished creating <default-ubuntu-1404> (0m2.70s).
-----> Converging <default-ubuntu-1404>...
       Preparing files for transfer
       Preparing dna.json
       Resolving cookbook dependencies with Berkshelf 5.1.0...
       Removing non-cookbook files before transfer
       Preparing data_bags
       Preparing secret
       Preparing validation.pem
       Preparing client.rb
-----> Installing Chef Omnibus (12.8.1)
       Downloading https://omnitruck.chef.io/install.sh to file /tmp/install.sh
       Trying wget...
       Download complete.
       ubuntu 14.04 x86_64
       Getting information for chef stable 12.8.1 for ubuntu...
       downloading https://omnitruck.chef.io/stable/chef/metadata?v=12.8.1&p=ubuntu&pv=14.04&m=x86_64
         to file /tmp/install.sh.15/metadata.txt
       trying wget...
       sha1 d202d57512b6a2d795d0fb6df566ca906745c7d8
       sha256   92b7f3eba0a62b20eced2eae03ec2a5e382da4b044c38c20d2902393683c77f7
       url  https://packages.chef.io/files/stable/chef/12.8.1/ubuntu/14.04/chef_12.8.1-1_amd64.deb
       version  12.8.1
       downloaded metadata file looks valid...
       downloading https://packages.chef.io/files/stable/chef/12.8.1/ubuntu/14.04/chef_12.8.1-1_amd64.deb
         to file /tmp/install.sh.15/chef_12.8.1-1_amd64.deb
       trying wget...
       Comparing checksum with sha256sum...
       Installing chef 12.8.1
       installing with dpkg...
       Selecting previously unselected package chef.
(Reading database ... 16023 files and directories currently installed.)
       Preparing to unpack .../chef_12.8.1-1_amd64.deb ...
       Unpacking chef (12.8.1-1) ...
       Setting up chef (12.8.1-1) ...
       Thank you for installing Chef!
       Transferring files to <default-ubuntu-1404>
       Starting Chef Client, version 12.8.1
       Creating a new client identity for default-ubuntu-1404 using the validator key.
       resolving cookbooks for run list: ["cc_sumologic-collector::collector"]
       Synchronizing Cookbooks:
         - cc_sumologic-collector (0.32.0)
         - sumologic-collector (1.2.21)
       Installing Cookbook Gems:
       Compiling Cookbooks...
       [2017-06-12T21:00:28+00:00] WARN: Cloning resource attributes for service[collector] from prior resource (CHEF-3694)
       [2017-06-12T21:00:28+00:00] WARN: Previous service[collector]: /tmp/kitchen/cache/cookbooks/cc_sumologic-collector/recipes/collector.rb:82:in `from_file'
       [2017-06-12T21:00:28+00:00] WARN: Current  service[collector]: /tmp/kitchen/cache/cookbooks/cc_sumologic-collector/recipes/collector.rb:91:in `from_file'
       Converging 22 resources
       Recipe: cc_sumologic-collector::collector
         * directory[/etc] action create (up to date)
         * template[/etc/sumo.conf] action create
           - create new file /etc/sumo.conf
           - update content in file /etc/sumo.conf from none to 22624b
           - suppressed sensitive resource
           - change mode from '' to '0600'
           - change owner from '' to 'root'
           - change group from '' to 'root'
         * directory[/etc/sumo.d] action create
           - create new directory /etc/sumo.d
           - change mode from '' to '0755'
           - change owner from '' to 'root'
           - change group from '' to 'root'
       Recipe: cc_sumologic-collector::_auth_logs
         * sumo_source_local_file[localfile-authlog] action create[2017-06-12T21:00:28+00:00] WARN: /tmp/kitchen/cache/cookbooks/sumologic-collector/libraries/provider_source.rb:25:in `block (3 levels) in <class:SumoSource>': property owner is declared in both file[/etc/sumo.d/localfile-authlog.json] and #<Chef::Provider::SumoSourceLocalFile:0x00000004eec920>. Use new_resource.owner instead. At /tmp/kitchen/cache/cookbooks/sumologic-collector/libraries/provider_source.rb:25:in `block (3 levels) in <class:SumoSource>'
       [2017-06-12T21:00:28+00:00] WARN: /tmp/kitchen/cache/cookbooks/sumologic-collector/libraries/provider_source.rb:26:in `block (3 levels) in <class:SumoSource>': property group is declared in both file[/etc/sumo.d/localfile-authlog.json] and #<Chef::Provider::SumoSourceLocalFile:0x00000004eec920>. Use new_resource.group instead. At /tmp/kitchen/cache/cookbooks/sumologic-collector/libraries/provider_source.rb:26:in `block (3 levels) in <class:SumoSource>'
       [2017-06-12T21:00:28+00:00] WARN: /tmp/kitchen/cache/cookbooks/sumologic-collector/libraries/provider_source.rb:27:in `block (3 levels) in <class:SumoSource>': property mode is declared in both file[/etc/sumo.d/localfile-authlog.json] and #<Chef::Provider::SumoSourceLocalFile:0x00000004eec920>. Use new_resource.mode instead. At /tmp/kitchen/cache/cookbooks/sumologic-collector/libraries/provider_source.rb:27:in `block (3 levels) in <class:SumoSource>'
       [2017-06-12T21:00:28+00:00] WARN: /tmp/kitchen/cache/cookbooks/sumologic-collector/libraries/provider_source.rb:28:in `block (3 levels) in <class:SumoSource>': property checksum is declared in both file[/etc/sumo.d/localfile-authlog.json] and #<Chef::Provider::SumoSourceLocalFile:0x00000004eec920>. Use new_resource.checksum instead. At /tmp/kitchen/cache/cookbooks/sumologic-collector/libraries/provider_source.rb:28:in `block (3 levels) in <class:SumoSource>'

           - Create /etc/sumo.d/localfile-authlog.json
           * file[/etc/sumo.d/localfile-authlog.json] action create
             - create new file /etc/sumo.d/localfile-authlog.json
             - update content in file /etc/sumo.d/localfile-authlog.json from none to 8efe3b
             --- /etc/sumo.d/localfile-authlog.json 2017-06-12 21:00:28.372419575 +0000
             +++ /etc/sumo.d/.chef-localfile-authlog.json20170612-106-1c94jd32017-06-12 21:00:28.372419575 +0000
             @@ -1 +1,12 @@
             +{
             +  "api.version": "v1",
             +  "source": {
             +    "sourceType": "LocalFile",
             +    "name": "localfile-authlog",
             +    "description": "Auth Logs",
             +    "category": "_default/auth/",
             +    "hostName": "87e2d7a9159a",
             +    "pathExpression": "/var/log/auth.log"
             +  }
             +}

       Recipe: cc_sumologic-collector::_cc_service_logs
         * sumo_source_local_file[localfile-microservice] action create (skipped due to only_if)
       Recipe: cc_sumologic-collector::_chef_bootstrap_logs
         * sumo_source_local_file[localfile-chef-bootstrap] action create (skipped due to only_if)
       Recipe: cc_sumologic-collector::_chef_client_logs
         * sumo_source_local_file[localfile-chef-client] action create (skipped due to only_if)
       Recipe: cc_sumologic-collector::_consul_logs
         * sumo_source_local_file[localfile-consul] action create (skipped due to only_if)
       Recipe: cc_sumologic-collector::_es_logs
         * sumo_source_local_file[localfile-es] action create (skipped due to only_if)
       Recipe: cc_sumologic-collector::_nginx_logs
         * sumo_source_local_file[localfile-nginx_access] action create (skipped due to only_if)
         * sumo_source_local_file[localfile-nginx_proxy] action create (skipped due to only_if)
         * sumo_source_local_file[localfile-nginx_error] action create (skipped due to only_if)
       Recipe: cc_sumologic-collector::_rabbitmq_server_logs
         * sumo_source_local_file[localfile-rabbitmq-server] action create (skipped due to only_if)
       Recipe: cc_sumologic-collector::_syslog
         * sumo_source_local_file[localfile-syslog] action create[2017-06-12T21:00:28+00:00] WARN: /tmp/kitchen/cache/cookbooks/sumologic-collector/libraries/provider_source.rb:25:in `block (3 levels) in <class:SumoSource>': property owner is declared in both file[/etc/sumo.d/localfile-syslog.json] and #<Chef::Provider::SumoSourceLocalFile:0x00000004e2a460>. Use new_resource.owner instead. At /tmp/kitchen/cache/cookbooks/sumologic-collector/libraries/provider_source.rb:25:in `block (3 levels) in <class:SumoSource>'
       [2017-06-12T21:00:28+00:00] WARN: /tmp/kitchen/cache/cookbooks/sumologic-collector/libraries/provider_source.rb:26:in `block (3 levels) in <class:SumoSource>': property group is declared in both file[/etc/sumo.d/localfile-syslog.json] and #<Chef::Provider::SumoSourceLocalFile:0x00000004e2a460>. Use new_resource.group instead. At /tmp/kitchen/cache/cookbooks/sumologic-collector/libraries/provider_source.rb:26:in `block (3 levels) in <class:SumoSource>'
       [2017-06-12T21:00:28+00:00] WARN: /tmp/kitchen/cache/cookbooks/sumologic-collector/libraries/provider_source.rb:27:in `block (3 levels) in <class:SumoSource>': property mode is declared in both file[/etc/sumo.d/localfile-syslog.json] and #<Chef::Provider::SumoSourceLocalFile:0x00000004e2a460>. Use new_resource.mode instead. At /tmp/kitchen/cache/cookbooks/sumologic-collector/libraries/provider_source.rb:27:in `block (3 levels) in <class:SumoSource>'
       [2017-06-12T21:00:28+00:00] WARN: /tmp/kitchen/cache/cookbooks/sumologic-collector/libraries/provider_source.rb:28:in `block (3 levels) in <class:SumoSource>': property checksum is declared in both file[/etc/sumo.d/localfile-syslog.json] and #<Chef::Provider::SumoSourceLocalFile:0x00000004e2a460>. Use new_resource.checksum instead. At /tmp/kitchen/cache/cookbooks/sumologic-collector/libraries/provider_source.rb:28:in `block (3 levels) in <class:SumoSource>'

           - Create /etc/sumo.d/localfile-syslog.json
           * file[/etc/sumo.d/localfile-syslog.json] action create
             - create new file /etc/sumo.d/localfile-syslog.json
             - update content in file /etc/sumo.d/localfile-syslog.json from none to 7dbc3f
             --- /etc/sumo.d/localfile-syslog.json  2017-06-12 21:00:28.376419575 +0000
             +++ /etc/sumo.d/.chef-localfile-syslog.json20170612-106-forjk1 2017-06-12 21:00:28.376419575 +0000
             @@ -1 +1,19 @@
             +{
             +  "api.version": "v1",
             +  "source": {
             +    "sourceType": "LocalFile",
             +    "name": "localfile-syslog",
             +    "description": "Syslog",
             +    "category": "_default/syslog/",
             +    "hostName": "87e2d7a9159a",
             +    "filters": [
             +      {
             +        "filterType": "Exclude",
             +        "name": "Exlude dhcp",
             +        "regexp": "*.DHCP.*"
             +      }
             +    ],
             +    "pathExpression": "/var/log/syslog"
             +  }
             +}

       Recipe: cc_sumologic-collector::_vpn
         * sumo_source_local_file[localfile-openvpn] action create (skipped due to only_if)
         * sumo_source_local_file[localfile-openvpn-as] action create (skipped due to only_if)
       Recipe: cc_sumologic-collector::_host_metrics
         * template[/etc/sumo.d/systemstats-default.json] action create
           - create new file /etc/sumo.d/systemstats-default.json
           - update content in file /etc/sumo.d/systemstats-default.json from none to e50096
           --- /etc/sumo.d/systemstats-default.json 2017-06-12 21:00:28.380419575 +0000
           +++ /etc/sumo.d/.chef-systemstats-default.json20170612-106-1urtm7c2017-06-12 21:00:28.380419575 +0000
           @@ -1 +1,14 @@
           +{
           +    "api.version": "v1",
           +    "source":
           +        {
           +            "name": "systemstats-default",
           +            "sourceType": "SystemStats",
           +            "category": "_default/metrics/systemstats/",
           +            "hostName": "87e2d7a9159a",
           +            "description": "Host Metrics",
           +              "metrics": ["CPU_Idle", "CPU_Stolen", "CPU_LoadAvg_1min", "CPU_LoadAvg_5min", "CPU_LoadAvg_15min", "Mem_UsedPercent", "Mem_FreePercent", "TCP_InboundTotal", "TCP_OutboundTotal", "Net_InPackets", "Net_OutPackets", "Net_InBytes", "Net_OutBytes", "Disk_ReadBytes", "Disk_WriteBytes", "Disk_Queue", "Disk_InodesAvailable", "Disk_Used", "Disk_Available"],
           +            "interval": 60000
           +        }
           +}
       Recipe: cc_sumologic-collector::_graphite_metrics
         * template[/etc/sumo.d/graphite-default.json] action create (skipped due to only_if)
       Recipe: cc_sumologic-collector::collector
         * remote_file[/tmp/kitchen/cache/sumocollector.deb] action create
           - create new file /tmp/kitchen/cache/sumocollector.deb
           - update content in file /tmp/kitchen/cache/sumocollector.deb from none to e14989
           (file sizes exceed 10000000 bytes, diff output suppressed)
         * dpkg_package[sumocollector] action install
           - install version 1:19.182-25 of package sumocollector
         * service[collector] action enable (up to date)
         * service[collector] action start (up to date)
         * service[collector] action restart
           - restart service service[collector]

       Running handlers:
       Running handlers complete
       Chef Client finished, 10/25 resources updated in 24 seconds
       Finished converging <default-ubuntu-1404> (0m41.73s).
-----> Kitchen is finished. (0m44.86s)

Berksfile:

$ grep sumo Berksfile*
Berksfile:cookbook 'sumologic-collector', github: 'SumoLogic/sumologic-collector-chef-cookbook', branch: 'master'
Berksfile.lock:  cc_sumologic-collector
Berksfile.lock:  sumologic-collector
Berksfile.lock:    git: https://github.com/SumoLogic/sumologic-collector-chef-cookbook.git
Berksfile.lock:  cc_sumologic-collector (0.32.0)
Berksfile.lock:    sumologic-collector (>= 0.0.0)
Berksfile.lock:  sumologic-collector (1.2.21)
majormoses commented 7 years ago

I spoke with @duchatran and this is a non issue.