config tests fail on output expectation

[vdice]

Encountered the following failure during an e2e run which appears to be a case of indeterminate ordering of expected command output.

------------------------------
$ deis auth:register http://deis.k8s.local --username=test-36224 --password=asdf1234 --email=test-36224@deis.io
Registered test-36224
Logged in as test-36224
$ deis apps:create test-443180143 --no-remote
Creating Application... done, created test-443180143
If you want to add a git remote for this app later, use `deis git:remote -a test-443180143`
$ deis config:set DEIS_KUBERNETES_DEPLOYMENTS=1 --app=test-443180143
Creating config... done

=== test-443180143 Config
DEIS_KUBERNETES_DEPLOYMENTS      1
$ deis builds:create --app=test-443180143 deis/example-dockerfile-http
Creating build... done
$ deis config:set FOO=null BAR=nil -a test-443180143
Creating config... done

=== test-443180143 Config
BAR                              nil
DEIS_KUBERNETES_DEPLOYMENTS      1
FOO                              null
$ deis config:list -a test-443180143
=== test-443180143 Config
BAR                              nil
DEIS_KUBERNETES_DEPLOYMENTS      1
FOO                              null
$ deis apps:destroy --app=test-443180143 --confirm=test-443180143
Destroying test-443180143...
done in 0s
$ deis auth:cancel --username=test-36224 --password=asdf1234 --yes
Please log in again in order to cancel this account
Logged in as test-36224
Account cancelled

• Failure [78.304 seconds]
deis config
/go/src/github.com/deis/workflow-e2e/tests/deployments_config_test.go:284
  with an existing user
  /go/src/github.com/deis/workflow-e2e/tests/deployments_config_test.go:237
    who owns an existing app that has already been deployed
    /go/src/github.com/deis/workflow-e2e/tests/deployments_config_test.go:235
      that user can set multiple environment variables at once on that app [It]
      /go/src/github.com/deis/workflow-e2e/vendor/github.com/onsi/ginkgo/ginkgo_dsl.go:354

      Expected
          <string>: === test-443180143 Config

      to match regular expression
          <string>: FOO\s+null

      /go/src/github.com/deis/workflow-e2e/tests/deployments_config_test.go:90
------------------------------

The spec as currently written expects FOO before BAR via:

sess, err := cmd.Start("deis config:set FOO=null BAR=nil -a %s", &user, app.Name)
...
output := string(sess.Out.Contents())
Expect(output).To(MatchRegexp(`FOO\s+null`))
Expect(output).To(MatchRegexp(`BAR\s+nil`))

Which I seems to bring with it inherent order.

Also, I believe sess.Out.Contents() should be sess.Wait().Out.Contents() to fully wait for the process to end, per the very last section in https://onsi.github.io/gomega/#gexec-testing-external-processes

[vdice]

Not able to reproduce this when running locally so unable to verify a proposed fix... keeping this open for now in case it is hit again.

[Joshua-Anderson]

The solution for this problem is to sort CLI output. Right now the SDK supports sorting, so all it will take is a PR on workflow-cli to sort the output.

[helgi]

Has this been done in the CLI yet?

[bacongobbler]

I believe so, yes. https://github.com/deis/workflow-cli/blob/a6cd7e6a4cc73c1999e879cd3e6f6918abedf9c4/cmd/config.go#L32 seems to indicate that as such. I think we can close this one out.