search

smartystreets / goconvey

Go testing in the browser. Integrates with `go test`. Write behavioral tests in Go.
http://smartystreets.github.io/goconvey/
Other
8.25k stars 555 forks source link

Broken tests! #328

Open mdwhatcott opened 9 years ago

mdwhatcott commented 9 years ago

This should actually be resolved with this pull request:

smartystreets/goconvey: Pull Request #317

We'll just have to see when 1.5 ships.

This group of tests fail randomly (I consistently get at least one failure).

=== RUN   TestWatcher-4

  Subject: Watcher operations 
    Initial scan results 2015/06/12 17:45:59.114291 integration.go:99: Stopping the watcher...
✘
    Manual execution produces additional results 2015/06/12 17:45:59.114864 integration.go:95: Gathering folders for immediate execution...
2015/06/12 17:45:59.115173 integration.go:99: Stopping the watcher...
✘
    Ignore and Reinstate commands are not reflected in the scan results 2015/06/12 17:45:59.115632 integration.go:73: Ignoring specified folders
2015/06/12 17:45:59.115681 integration.go:161: Currently ignored folders: map[/tmp/goconvey182352595/integration_testing/sub:{}]
2015/06/12 17:45:59.115861 integration.go:80: Reinstating specified folders
2015/06/12 17:45:59.115980 integration.go:99: Stopping the watcher...
✘
    Adjusting the root changes the number of folders in the scanned results 2015/06/12 17:45:59.116323 integration.go:68: Adjusting root...
2015/06/12 17:45:59.366617 integration.go:68: Adjusting root...
2015/06/12 17:45:59.366993 integration.go:99: Stopping the watcher...
✘
    A bogus command does not cause any additional scanning beyond the initial scan 2015/06/12 17:45:59.367648 integration.go:104: Unrecognized command from server: UNKNOWN IN
STRUCTION
2015/06/12 17:45:59.367807 integration.go:120: File system state modified, publishing current folders... 0 1434149363
2015/06/12 17:45:59.618026 integration.go:99: Stopping the watcher...
✔
    Scanning occurs as a result of a file system update 2015/06/12 17:45:59.618874 integration.go:120: File system state modified, publishing current folders... 0 1434149363
2015/06/12 17:46:00.621113 integration.go:120: File system state modified, publishing current folders... 1434149363 1434149364
2015/06/12 17:46:01.623002 integration.go:99: Stopping the watcher...
✔
    Scanning does not occur as a result of resuming after a pause 2015/06/12 17:46:01.623488 integration.go:87: Pausing watcher...
2015/06/12 17:46:01.873707 integration.go:91: Resuming watcher...
2015/06/12 17:46:01.874162 integration.go:120: File system state modified, publishing current folders... 0 1434149364
2015/06/12 17:46:02.124398 integration.go:99: Stopping the watcher...
✔
    Scanning should not occur when the watcher is paused 2015/06/12 17:46:02.125197 integration.go:120: File system state modified, publishing current folders... 0 1434149364
2015/06/12 17:46:02.375410 integration.go:87: Pausing watcher...
2015/06/12 17:46:03.626461 integration.go:91: Resuming watcher...
2015/06/12 17:46:03.626991 integration.go:120: File system state modified, publishing current folders... 1434149364 1434149367
2015/06/12 17:46:03.877245 integration.go:99: Stopping the watcher...
✔

Failures:

  * /opt/home/mike/go/src/github.com/smartystreets/goconvey/web/server/watch/integration_test.go 
  Line 77:
  Expected: '1'
  Actual:   '0'
  (Should be equal)

  * /opt/home/mike/go/src/github.com/smartystreets/goconvey/web/server/watch/integration_test.go 
  Line 91:
  Expected: '2'
  Actual:   '1'
  (Should be equal)

  * /opt/home/mike/go/src/github.com/smartystreets/goconvey/web/server/watch/integration_test.go 
  Line 108:
  Expected: '1'
  Actual:   '0'
  (Should be equal)

  * /opt/home/mike/go/src/github.com/smartystreets/goconvey/web/server/watch/integration_test.go 
  Line 124:
  Expected: '3'
  Actual:   '2'
  (Should be equal)

65 total assertions

--- FAIL: TestWatcher-4 (4.77s)

Possible problems looming:

msteinert commented

$ go version
go version devel +06ef022 Thu Jun 11 18:39:14 2015 +0000 linux/amd64
$ git branch
* master
This test fails consistently:

=== RUN   TestExecutor-4

  Subject: Execution of test packages and aggregation of parsed results 
    When tests packages are executed 
      The result should include parsed results for each test package. ✔
    When the executor is idle 
      The status of the executor should be 'idle' ✔
    When the status is updated 
      The status flag should be set to true ✔
    During test execution 
      The status of the executor should be 'executing' ✘

Failures:

  * /opt/home/mike/go/src/github.com/smartystreets/goconvey/web/server/executor/executor_test.go 
  Line 41:
  Expected: 'executing'
  Actual:   'idle'
  (Should be equal)

4 total assertions

--- FAIL: TestExecutor-4 (0.03s)
=== RUN   TestConcurrentTester-4
dlogvinenko commented 9 years ago

Here are my results for goconvey/convey tests

➜  convey git:(master) go version
go version go1.5.1 darwin/amd64
➜  convey git:(master) git branch
* master
➜  convey git:(master) go test
PASS
ok      github.com/smartystreets/goconvey/convey    0.030s

So I would change title of this ticket if you mean another tests.

mdwhatcott commented 9 years ago

Here's the package with the failing tests:

go test -v github.com/smartystreets/goconvey/web/server/watch
jasdel commented 8 years ago

I'm also seeing this issue, with "github.com/smartystreets/goconvey/web/server/watch/integration_test.go", but i'm consistently getting five failed assertions.

--- PASS: TestSum (0.00s)
=== RUN   TestWatcher

  Subject: Watcher operations
    Initial scan results ✘
    Manual execution produces additional results ✘
    Ignore and Reinstate commands are not reflected in the scan results ✘
    Adjusting the root changes the number of folders in the scanned results ✘
    A bogus command does not cause any additional scanning beyond the initial scan ✔
    Scanning occurs as a result of a file system update ✔
    Scanning does not occur as a result of resuming after a pause ✔
    Scanning should not occur when the watcher is paused ✘
Failures:

  * /tmp/golang/src/github.com/smartystreets/goconvey/web/server/watch/integration_test.go
  Line 81:
  Expected: '1'
  Actual:   '0'
  (Should be equal)

  * /tmp/golang/src/github.com/smartystreets/goconvey/web/server/watch/integration_test.go
  Line 95:
  Expected: '2'
  Actual:   '1'
  (Should be equal)

  * /tmp/golang/src/github.com/smartystreets/goconvey/web/server/watch/integration_test.go
  Line 112:
  Expected: '1'
  Actual:   '0'
  (Should be equal)

  * /tmp/golang/src/github.com/smartystreets/goconvey/web/server/watch/integration_test.go
  Line 128:
  Expected: '3'
  Actual:   '2'
  (Should be equal)

  * /tmp/golang/src/github.com/smartystreets/goconvey/web/server/watch/integration_test.go
  Line 196:
  Expected: '2'
  Actual:   '1'
  (Should be equal)