search

opencontainers / image-tools

OCI Image Tooling
https://opencontainers.org
Apache License 2.0
271 stars 82 forks source link

What flavor tests do we want? #18

Open philips opened 8 years ago

philips commented 8 years ago

From @wking on September 7, 2016 9:57

257 shows we could really use a test suite. I'm flexible, but it feels like our command line API is more stable than our current Go APIs. How would folks feel about a test harness exercising the command line API? Do we have a sharness/bats preference (I have more experience with sharness, but runC uses bats)? Should this wait until after opencontainers/tob#18 (is there an estimated timeframe for that decision)?

Copied from original issue: opencontainers/image-spec#272

philips commented 8 years ago

From @glestaris on September 7, 2016 13:24

I agree that the CLI should be tested instead of any Go API. I would prefer if we could use a Go testing framework, even though it's a CLI. Ginkgo [1] is being used in Cloud Foundry for all types of testing (API/CLI/unit).

Also, I guess this issue is mainly targeting a CLI test suite, but, would it make sense to consider having unit tests on some of the most complex operations?

[1] http://onsi.github.io/ginkgo/

philips commented 8 years ago

From @wking on September 7, 2016 15:49

On Wed, Sep 07, 2016 at 06:24:42AM -0700, George Lestaris wrote:

I would prefer if we could use a Go testing framework, even though it's a CLI. Ginkgo [1] is being used in Cloud Foundry for all types of testing (API/CLI/unit).

Ginko looks nice, but I prefer shell-oriented test harnesses for the command line because there's less boilerplate around invocation. However, any framework around the command line is better than none, so if maintainers want tests in Ginko (or whatever), I'll add them in that framework.

Also, I guess this issue is mainly targeting a CLI test suite, but, would it make sense to consider having unit tests on some of the most complex operations?

As things currently stand, I think ‘oci-image-tool unpack SRC DEST’ is more likely to be around in two months than manifest.unpack(walker, dest), and testing I'd rather test the more stable interface. Testing the command line API is also going to cover cmd/oci-image-tool/unpack.go, and I'd rather have coverage first. We can fill in unit tests later to make it easier to isolate errors and avoid combinatoric issues ;). But again, if maintainers want Go tests in image/*_test.go first, I'll go that way.