inspec / train

Transport Interface to unify communication over SSH, WinRM, and friends.
Apache License 2.0
119 stars 87 forks source link

Use Ohai for platform detection #331

Open clintoncwolfe opened 6 years ago

clintoncwolfe commented 6 years ago

Description

Opened on behalf of @cheeseplus

1 thing I'd ask for is compat with Ohai/mixlib/chef system detection semantics (we currently have a method called train_to_mixlib) (related it's super confusing for folks writing inspec tests for their chef cookbooks given the nefariously subtle differences) (edited)

cwolfe [2:28 PM] I'm hearing that you want a more compatible (not neccisarily more extensible) detection API, though (edited)

cheeseplus [2:33 PM] mostly just the train system discovery aligning (recognizing this is potentially a major version change) and bastion support (which seems to be already on the way) we don't really touch anything besides "pure" transports like SSH and WinRM at this time and we're not really looking to leverage much $CLOUD aside from say support for grabbing MSI creds in Azure Cloud Shell the aligning for platform stuff goes a long way to writing simpler code - stuff like opensuse vs suse vs sles and treating platform and platform family similarly (platform family is an array in train but a single string in Ohai)

Train and Platform Version

1.4.x

frezbo commented 6 years ago

@clintoncwolfe we already have bastion support

arlimus commented 6 years ago

From my current understanding: train and Ohai overlap in a number of fields. However, they are different in the way that train targets remote + non-OS systems and Ohai is very closely tied to Chef's ecosystem and local. My initial reaction is to keep Train free from being tied to Ohai, i.e. would argue in this point against using Ohai and sticking to train's native detection instead.

This may be one of those topics where it's best to get into a discussion room and have a talk about it, which is what I'd love to propose here. It would also help us explain the differences and commonalities.

Could Ohai possibly use Train's mechanism? (preposterous) food for thought :wink: cc @cheeseplus

"the aligning for platform stuff goes a long way to writing simpler code - stuff like opensuse vs suse vs sles and treating platform and platform family similarly" <= i like this :+1: . Also let's talk about the differences and why they are

cheeseplus commented 6 years ago

I don't really want Ohai in Train or the other way around in terms of technical implementation, so much as some way that naming of platform, platform family, and a number of similar and commonly used keys being compatible. The lower the overhead when switching between the frameworks will make us and users more productive. This also isn't just Train as mixlib-install and omnitruck each maintain their own mappings so my hope is to collapse as much of it as possible to be common.

From where we are sitting inside the greater Chef ecosystem, folks have to learn both the inspec/train and ohai platform nuances to write meaningful tests in the common case. Moreover, we already have too much divergence in things like mixlib-install, the older install.sh script, Ohai, and Train which ends up making it harder for us internally but also confuses users regularly.

Even if Train had a mapping that published something like ohai_platform or similar would be progress and help eliminate ugly mapping code elsewhere. I certainly don't want to lock InSpec unnecessarily to things in the Chef ecosystem but given it's the most commonly used config management in conjunction with it, I think it makes sense to smooth some of the rougher edges and subtle differences.

Example code that got me thinking "wow this is awful":

https://github.com/chef/chef-apply/blob/master/lib/chef_apply/action/install_chef/base.rb#L70-L96

    def train_to_mixlib(platform)
      opts = {
        platform_version: platform.release,
        platform: platform.name,
        architecture: platform.arch,
        product_name: "chef",
        product_version: :latest,
        channel: :stable,
        platform_version_compatibility_mode: true
      }
      case platform.name
      when /windows/
        opts[:platform] = "windows"
      when "redhat", "centos"
        opts[:platform] = "el"
      when "suse"
        opts[:platform] = "sles"
      when "amazon"
        opts[:platform] = "el"
        if platform.release.to_i > 2010 # legacy Amazon version 1
          opts[:platform_version] = "6"
        else
          opts[:platform_version] = "7"
        end
      end
      opts
    end
arlimus commented 6 years ago

Yep, fully on board. Let's get them closer to each other if we can. I'm really curious where differences are and what we can learn from both projects. A mapping sounds ok. This is where we should talk :+1:

clintoncwolfe commented 6 years ago

I just want to advocate for having a public (as in, on this issue) discussion about this, or at least recording the outcome of any internal discussions here.

cheeseplus commented 6 years ago

Adding some examples

centos / rhel
node['platform_family'] = 'rhel'
os.platform_family = 'redhat'

opensuse
node['platform'] = 'opensuseleap'
os.platform = 'opensuse'

amazon 2 linux
node['platform_family'] = 'amazon'
os.platform_family = 'redhat'
tas50 commented 6 years ago

I figured I should chime in here. If it would make things easier we could probably break out the platform detection from ohai into a gem so we don't need to bring in the whole ohai gem just for platform detection. Train and Ohai could both just dep on the common gem.

tas50 commented 6 years ago

Also I entirely apologize for opensuseleap as a platform in Ohai. We should have just left that one as opensuse. Too late now :(

lamont-granquist commented 6 years ago

I have never understood why inspec took ohai's platform and platform_family concepts and then changed the labels around producing an xkcd/927 problem, which is poor user experience.

chris-rock commented 6 years ago

InSpec inherited the naming from Serverspec to be compatible with existing Serverspec test scripts, it never invented new ones.

zenspider commented 4 years ago

I think this is a no-go, since ohai's is local only and train's is local/remote and the values between them don't line up. If we were to extract the two into their own separate gem, unification would have to happen first and all products that depend on ohai & train would have to adapt.

That is a lot to ask for where we're at. I'm leaning towards closing. Assigning to keka to weigh in.