Open peti opened 9 years ago
Thank you for testing. https://github.com/junjihashimoto/test-sandbox-compose/blob/master/.travis.yml#L27 Currently, test-sandbox-compose-daemon should run before cabal-test.
In other words, cabal test
is broken?
When setup is nothing before test, cabal test
is broken.
I improve test so that the setup is not needed.
I have improved test( add setup code to haskell-code). Version-bump this package.
I re-enabled the test suite for the current version, and the errors I used to get are now gone. Unfortunately, there's plenty of new errors, because the test suite has undeclared dependencies on nc
, etc.:
Failures:
1) Basic Test Run nc : port 12181
uncaught exception: IOException of type NoSuchThing (/tmp/nix-build-haskell-test-sandbox-0.1.3.drv-0/test_6970/ncfile16970: createProcess: runInteractiveProcess: exec: does not exist (No such file or directory))
2) Basic Test isBinable' 12181
expected: False
but got: True
3) Basic Test Test isBinable 12181
expected: False
but got: True
4) Basic Test Test sendTo
uncaught exception: IOException of type NoSuchThing (/tmp/nix-build-haskell-test-sandbox-0.1.3.drv-0/test_6970/ncfile6970: createProcess: runInteractiveProcess: exec: does not exist (No such file or directory))
5) Basic Test : run
uncaught exception: IOException of type NoSuchThing (/tmp/nix-build-haskell-test-sandbox-0.1.3.drv-0/test_6971/echoecho6970: createProcess: runInteractiveProcess: exec: does not exist (No such file or directory))
6) signal test send kill signal for process groups
uncaught exception: IOException of type NoSuchThing (/tmp/nix-build-haskell-test-sandbox-0.1.3.drv-0/test_6971/str26970: createProcess: runInteractiveProcess: exec: does not exist (No such file or directory))
7) signal test send kill signal for process groups
uncaught exception: IOException of type NoSuchThing (/tmp/nix-build-haskell-test-sandbox-0.1.3.drv-0/test_6971/str26970: createProcess: runInteractiveProcess: exec: does not exist (No such file or directory))
8) signal test not send kill signal for process groups
uncaught exception: IOException of type NoSuchThing (/tmp/nix-build-haskell-test-sandbox-0.1.3.drv-0/test_6971/str26970: createProcess: runInteractiveProcess: exec: does not exist (No such file or directory))
I'm not sure whether it's worth trying to fix these issues. It appears that the test suite is based on assumptions that really don't mix well with a batch-oriented CI system, like NixOS uses. It might be easiest to just keep the test suite disabled and to trust that users of the packaged will run it manually.
Duh, I just realized that I ran the tests for the wrong package! Those results are for test-sandbox
, of course. For test-sandbox-composable
, the outcome looks like this:
Starting process daemon... InvalidYaml (Just (YamlException "Yaml file not found: sample/test-sandbox-compose.yml"))
Process daemon not running.
- command-line: /tmp/nix-build-haskell-test-sandbox-compose-0.1.3.drv-0/test-sandbox-compose-0.1.3/dist/build/test-sandbox-compose/test-sandbox-compose daemon --conf sample/test-sandbox-compose.yml
- exit code: ExitFailure 1
run daemon FAILED [1]
Server Test
help
up FAILED [2]
kill FAILED [3]
up FAILED [4]
destroy FAILED [5]
Failures:
1) run daemon run daemon
uncaught exception: IOException of type UserError (user error (Process daemon not running.
- command-line: /tmp/nix-build-haskell-test-sandbox-compose-0.1.3.drv-0/test-sandbox-compose-0.1.3/dist/build/test-sandbox-compose/test-sandbox-compose daemon --conf sample/test-sandbox-compose.yml
- exit code: ExitFailure 1))
2) Server Test up
expected: ""
but got: "test-sandbox-compose: FailedConnectionException2 \"localhost\" 16968 False connect: does not exist (Connection refused)\n"
3) Server Test kill
expected: ""
but got: "test-sandbox-compose: FailedConnectionException2 \"localhost\" 16968 False connect: does not exist (Connection refused)\n"
4) Server Test up
expected: ""
but got: "test-sandbox-compose: FailedConnectionException2 \"localhost\" 16968 False connect: does not exist (Connection refused)\n"
5) Server Test destroy
expected: ""
but got: "test-sandbox-compose: FailedConnectionException2 \"localhost\" 16968 False connect: does not exist (Connection refused)\n"
Randomized with seed 1544686696
Anyway, the conclusions I drew earlier probably still apply.
Thank you for comments. I agree with your opinion that assumptions are not well for tests of the CI system. I try to create another nc-server to prevent manual test.
I replace nc with runhaskell and https://gist.github.com/junjihashimoto/42f6519e9e63f7d66347.
I tried to update the test like this https://github.com/junjihashimoto/test-sandbox-compose/pull/4. But this is failed. I realize this is bad idea. You opinion is right. I should change so that users of the packaged can run it manually.
Citing from http://hydra.cryp.to/build/599429/nixlog/1/raw: