[design] servpeek high level design

[milosgajdos]

This issue to discuss and share ideas on a high level design of servpeek tool.

The goal imo should not only be to just qa the OS images (built by either docker or packer), although that might be a good start, but also to provide an introspective "peek" into installed packages, image configuration files and services running inside the built OS image - i.e. inside the running OS/container. Something which was nicely outlined in this blog post.

There are two primary options to do this:

• declarative approach - define the things you want to check in some kind of declarative language like yaml or toml (or what have you) and then invoke the test on those against the image
• imperative approach - write the tests directly in Go and compile+run them as part of the image build time

Both have their cons and pros. Curious what other folks think of this.

[bryanwb]

definitely pro imperative approach here. It can grow in ways you couldn't have foreseen.

when you use yaml/toml/markdown, there are a couple more drawbacks:

1. Everything is a fucking string, it can be quite hard to predict whether a variable is a boolean true or "True"
2. Doing something the author didn't intend is quite hard
3. You have to run your code in order to find a syntax error, linters can do it for you, but it's easy to forget

[garethr]

A declarative interface would be nice, but YAML/TOML/etc. are data formats, not a declarative language. I'd start with the primatives in Go - people can always build higher level interfaces later (like in golog).

My main interest is in something that:

• uses a real unit test framework
• can compile binaries that specifiy a system that I can ship around

[milosgajdos]

Arguably the biggest pro of declarative approach is that such approach is language agnostic - imagine you build service which consumes json with test declaration that it then executes inside the container - obviously the json would be constructed from a declaration.

I agree with the YAML/TOML point. As for the real unit test framework - Go has a built in testing package which is part of the standard lib, so in theory this could at the end boil down to simple go test command i.e. you'd not even need a binary - merely a simple import of the core codebase or something along those lines, although go test would require Go tools being available inside the image, so shipping the binary might just be the option.

[bryanwb]

i would prefer to start imperative, you could always build support for toml/yaml later. fwiw, i have always preferred tools that support a specific prog language well, rather than one that supports multiple languages poorly

[garethr]

@milosgajdos83 so having an interface for describing tests which is serialisable/deserializable is useful. For instance building an HCL (https://github.com/hashicorp/hcl) interface might be interesting. But I think that's just one interface, not the tool. I can think of other tools as well that you could build ontop of this.

I think the go test model is nice, you can produce a portable executable which can run the tests with go test -o.

I think the bulk of the work is in what serverspec calls resources and what testinfra calls modules. Picking a small number of resources would be a good starting point, probably also picking a target platform or platforms for the first version (ie. Debian? Debian and Ubuntu, Debian and Centos, all RHEL variants, Windows, etc.)

[agonzalezro]

I would go for something imperative here. A declarative approach will be really opinionated and not worthy. Doing a tool to use servpeek in a declarative way (without knowing any Go) should be pretty straight forward.

Here you can find an example of what I think you wanted to see as a declarative format:

package main

import (
    "fmt"
    "log"

    "gopkg.in/yaml.v2"

    "github.com/milosgajdos83/servpeek/resource"
    "github.com/milosgajdos83/servpeek/resource/pkg"
)

const mockedFile = `---
packages:
  - name: neovim
    type: pip
  - name: mercurial
    type: pip
    version: 3.5.2`

type Tests map[string][]TestPackage

type TestPackage struct {
    Name, Version, Type string
}

func main() {
    var tests Tests

    if err := yaml.Unmarshal([]byte(mockedFile), &tests); err != nil {
        log.Fatal(err)
    }

    var (
        versionedTests   []resource.Pkg
        unVersionedTests []resource.Pkg
    )

    for _, test := range tests["packages"] {
        p := resource.Pkg{
            Name:    test.Name,
            Version: test.Version,
            Type:    test.Type,
        }

        if p.Version == "" {
            unVersionedTests = append(unVersionedTests, p)
        } else {
            versionedTests = append(versionedTests, p)
        }

        if err := pkg.IsInstalled(unVersionedTests...); err != nil {
            fmt.Println(err)
        }

        if err := pkg.IsInstalledVersion(versionedTests...); err != nil {
            fmt.Println(err)
        }
    }
}