search

instana / instana-agent-chef

Instana agent chef cookbook
https://www.instana.com/
Apache License 2.0
0 stars 8 forks source link

OS support #6

Closed mitom closed 6 years ago

mitom commented 6 years ago

Hi,

I've tried to use this cookbook to install the agent on a fairly wide range of OSs, all of which should be supported according to your docs with mixed success. All the instances are less than 2 months old on AWS from official images.

What worked: Ubuntu 16.04 Ubuntu 14.04 Amazon linux Centos 6 Debian 9

What didn't work: Redhat 7: Initially errored, after updating the nss package it was able to install and complete. This may not be related to the cookbook but it's certainly not handled nicely nor noted anywhere. Redhat 6: Errors:

[2018-07-19T10:50:11-04:00] INFO: execute[yum clean metadata Instana-Agent] ran successfully
      - execute yum clean metadata --disablerepo=* --enablerepo=Instana-Agent
[2018-07-19T10:50:11-04:00] INFO: template[/etc/yum.repos.d/Instana-Agent.repo] sending run action to execute[yum-makecache-Instana-Agent] (immediate)
    * execute[yum-makecache-Instana-Agent] action run[2018-07-19T10:50:11-04:00] INFO: Processing execute[yum-makecache-Instana-Agent] action run (/opt/chef/embedded/lib/ruby/gems/2.5.0/gems/chef-14.2.0/lib/chef/provider/yum_repository.rb line 60)

      [execute] https://_:****************@packages.instana.io/agent/generic/x86_64/repodata/repomd.xml: [Errno 14] problem making ssl connection
                Trying other mirror.
                Error: Cannot retrieve repository metadata (repomd.xml) for repository: Instana-Agent. Please verify its path and try again

      ================================================================================
      Error executing action `run` on resource 'execute[yum-makecache-Instana-Agent]'
      ================================================================================

      Mixlib::ShellOut::ShellCommandFailed
      ------------------------------------
      Expected process to exit with [0], but received '1'
      ---- Begin output of yum -q -y makecache --disablerepo=* --enablerepo=Instana-Agent ----
      STDOUT:
      STDERR: https://_:****************@packages.instana.io/agent/generic/x86_64/repodata/repomd.xml: [Errno 14] problem making ssl connection
      Trying other mirror.
      Error: Cannot retrieve repository metadata (repomd.xml) for repository: Instana-Agent. Please verify its path and try again
      ---- End output of yum -q -y makecache --disablerepo=* --enablerepo=Instana-Agent ----
      Ran yum -q -y makecache --disablerepo=* --enablerepo=Instana-Agent returned 1

      Resource Declaration:
      ---------------------
      # In /opt/chef/embedded/lib/ruby/gems/2.5.0/gems/chef-14.2.0/lib/chef/provider/yum_repository.rb

       60:         declare_resource(:execute, "yum-makecache-#{new_resource.repositoryid}") do
       61:           command "yum -q -y makecache --disablerepo=* --enablerepo=#{new_resource.repositoryid}"
       62:           action :nothing
       63:           only_if { new_resource.enabled }
       64:         end
       65:

      Compiled Resource:
      ------------------
      # Declared in /opt/chef/embedded/lib/ruby/gems/2.5.0/gems/chef-14.2.0/lib/chef/provider/yum_repository.rb:60:in `block in <class:YumRepository>'

      execute("yum-makecache-Instana-Agent") do
        action [:nothing]
        default_guard_interpreter :execute
        command "yum -q -y makecache --disablerepo=* --enablerepo=Instana-Agent"
        backup 5
        declared_type :execute
        cookbook_name "instana-agent"
        user nil
        domain nil
        only_if { #code block }
      end

      System Info:
      ------------
      chef_version=14.2.0
      platform=redhat
      platform_version=6.7
      ruby=ruby 2.5.1p57 (2018-03-29 revision 63029) [x86_64-linux]
      program_name=/usr/bin/chef-client
      executable=/opt/chef/bin/chef-client

[2018-07-19T10:50:12-04:00] INFO: Running queued delayed notifications before re-raising exception

    ================================================================================
    Error executing action `create` on resource 'yum_repository[Instana-Agent]'
    ================================================================================

    Mixlib::ShellOut::ShellCommandFailed
    ------------------------------------
    execute[yum-makecache-Instana-Agent] (/opt/chef/embedded/lib/ruby/gems/2.5.0/gems/chef-14.2.0/lib/chef/provider/yum_repository.rb line 60) had an error: Mixlib::ShellOut::ShellCommandFailed: Expected process to exit with [0], but received '1'
    ---- Begin output of yum -q -y makecache --disablerepo=* --enablerepo=Instana-Agent ----
    STDOUT:
    STDERR: https://_:****************@packages.instana.io/agent/generic/x86_64/repodata/repomd.xml: [Errno 14] problem making ssl connection
    Trying other mirror.
    Error: Cannot retrieve repository metadata (repomd.xml) for repository: Instana-Agent. Please verify its path and try again
    ---- End output of yum -q -y makecache --disablerepo=* --enablerepo=Instana-Agent ----
    Ran yum -q -y makecache --disablerepo=* --enablerepo=Instana-Agent returned 1

    Resource Declaration:
    ---------------------
    # In /var/chef/cache/cookbooks/instana-agent/recipes/system.rb

     37: yum_repository 'Instana-Agent' do
     38:    description 'The Agent repository by Instana, Inc.'
     39:    baseurl "#{domain}/agent/generic/x86_64"
     40:    gpgkey gpg_path
     41:    repo_gpgcheck true
     42:    gpgcheck false
     43:    action %i[create makecache]
     44:    only_if { %w[rhel suse amazon].include? node['platform_family'] }
     45: end
     46:

    Compiled Resource:
    ------------------
    # Declared in /var/chef/cache/cookbooks/instana-agent/recipes/system.rb:37:in `from_file'

    yum_repository("Instana-Agent") do
      action [:create, :makecache]
      updated true
      updated_by_last_action true
      default_guard_interpreter :default
      declared_type :yum_repository
      cookbook_name "instana-agent"
      recipe_name "system"
      baseurl "https://_:****************@packages.instana.io/agent/generic/x86_64"
      gpgkey "https://packages.instana.io/Instana.gpg"
      repo_gpgcheck true
      gpgcheck false
      repositoryid "Instana-Agent"
      only_if { #code block }
    end

    System Info:
    ------------
    chef_version=14.2.0
    platform=redhat
    platform_version=6.7
    ruby=ruby 2.5.1p57 (2018-03-29 revision 63029) [x86_64-linux]
    program_name=/usr/bin/chef-client
    executable=/opt/chef/bin/chef-client

[2018-07-19T10:50:12-04:00] INFO: Running queued delayed notifications before re-raising exception

Running handlers:
[2018-07-19T10:50:12-04:00] ERROR: Running exception handlers
  - CRChefReporting
Running handlers complete
[2018-07-19T10:50:12-04:00] ERROR: Exception handlers complete
Chef Client failed. 6 resources updated in 57 seconds
[2018-07-19T10:50:12-04:00] FATAL: Stacktrace dumped to /var/chef/cache/chef-stacktrace.out
[2018-07-19T10:50:12-04:00] FATAL: Please provide the contents of the stacktrace.out file if you file a bug report
[2018-07-19T10:50:12-04:00] FATAL: Mixlib::ShellOut::ShellCommandFailed: yum_repository[Instana-Agent] (instana-agent::system line 37) had an error: Mixlib::ShellOut::ShellCommandFailed: execute[yum-makecache-Instana-Agent] (/opt/chef/embedded/lib/ruby/gems/2.5.0/gems/chef-14.2.0/lib/chef/provider/yum_repository.rb line 60) had an error: Mixlib::ShellOut::ShellCommandFailed: Expected process to exit with [0], but received '1'
---- Begin output of yum -q -y makecache --disablerepo=* --enablerepo=Instana-Agent ----
STDOUT:
STDERR: https://_:****************@packages.instana.io/agent/generic/x86_64/repodata/repomd.xml: [Errno 14] problem making ssl connection
Trying other mirror.
Error: Cannot retrieve repository metadata (repomd.xml) for repository: Instana-Agent. Please verify its path and try again
---- End output of yum -q -y makecache --disablerepo=* --enablerepo=Instana-Agent ----
Ran yum -q -y makecache --disablerepo=* --enablerepo=Instana-Agent returned 1

This is the same error that Redhat 7 originally had, however updating nss did not solve it.

Centos 7: The agent installed but fails to start:

[root@ip-10-0-1-107 ~]# service instana-agent start
Starting Instana Agent
Not running.
Instana Agent hasn't started within the timeout allowed
please review file "/opt/instana/agent/data/log/instana-agent-console.log" to see the status of the service
[root@ip-10-0-1-107 ~]# cat /opt/instana/agent/data/log/instana-agent-console.log
null
Error occurred shutting down framework: java.lang.NumberFormatException: null
java.lang.NumberFormatException: null
    at java.lang.Integer.parseInt(Integer.java:542)
    at java.lang.Integer.parseInt(Integer.java:615)
    at org.apache.karaf.main.ConfigProperties.<init>(ConfigProperties.java:214)
    at org.apache.karaf.main.Main.updateInstancePidAfterShutdown(Main.java:228)
    at org.apache.karaf.main.Main.main(Main.java:193)

I figured it would support windows as the docs point to the instana docs for supported OSs but realised that's not the case. Is this something that's expected?

doertedev commented 6 years ago

Hey there. Thanks for engaging with us and giving this a spin.

First of all the mirror package.instana.io requires a modern TLS stack. In our previous experiences this particular case of an outdated NSS only occurred on SLES12 (hence the hint in our docs), but as you can see it can also happen in longer-running Instances of other enterprise distributions. The easiest way to test this would be curl the Oneliner Install method's script Url - https://setup.instana.io/agent - When this URl does not return a bash script via curl or another fetcher of web content, then the chances are high that your package manager isn't able to speak modern ciphers either.

What I'm more curious about is the fact that you're seeing an error message from our sysvinit script under a distribution which is being shipped with systemd. Can you elaborate if any customizations have been done on your side?

mitom commented 6 years ago

I've rebuilt the instances in question from scratch to double check, centos 7 - the issue you mentioned - got solved, so it may have been some difference. However, the redhat issue is now present on freshly spun up instances. It feels like that should not be the case.

Also, could you please react on whether windows support is expected or not?

doertedev commented 6 years ago

Can you please give me the output of openssl version on the RHEL instances? That sounds really odd. From when is the image you're using? AWS? GCP? Azure? Something homegrown?

The Windows MSI is downloadable via the User Management Portal, you can find more Informations here.

mitom commented 6 years ago

In both cases the version is OpenSSL 1.0.1e-fips 11 Feb 2013.

All of the instances are in AWS, the images in question are:

(eu-west-1 region for both).

doertedev commented 6 years ago

https://blog.pcisecuritystandards.org/migrating-from-ssl-and-early-tls

Indeed on June 30th the PCI SSC deprecated TLS1.0 - which keeps you from pinging our agent repo through this ssl version. Chef does appear to have customers running older infra, which is most likely why you're able to ping https://omnitruck.chef.io/ but not our loadbalancers with TLS1.0. So this is neither a chef nor a cookbook issue itself. yum upgrade nss curl openssl resolves this. Yet, if you want to distribute the packages to your RHEL instances and depend on older OpenSSL versions, feel free to proxy our package repo as described here.

Regards, Stefan.