search

pokt-network / pocket-core

Official implementation of the Pocket Network Protocol
http://www.pokt.network
MIT License
209 stars 103 forks source link

[BUG REPORT] tests are failing #1517

Closed lookfirst closed 1 year ago

lookfirst commented 1 year ago

Describe the bug check out staging branch, try to run tests.

To Reproduce

go test ./...
6274da626c20dac6d373bf7bc703e375de43f3ba
abb8a168df35c571e9503edabdc9f7959d8e58fc
--- FAIL: TestClaimAminoTx (51.08s)
    --- FAIL: TestClaimAminoTx/claim_tx_from_amino_with_amino_codec_ (51.08s)
        tx_test.go:1249: proof message was not received afterward
--- FAIL: TestClaimProtoTx (0.28s)
    --- FAIL: TestClaimProtoTx/claim_tx_from_a_proto_with_proto_codec (0.28s)
panic: failed to listen on 0.0.0.0:36657: listen tcp 0.0.0.0:36657: bind: address already in use [recovered]
    panic: failed to listen on 0.0.0.0:36657: listen tcp 0.0.0.0:36657: bind: address already in use

goroutine 30271 [running]:
testing.tRunner.func1.2({0x10159b580, 0x140017108a0})
    /opt/homebrew/Cellar/go/1.19.4/libexec/src/testing/testing.go:1396 +0x1c8
testing.tRunner.func1()
    /opt/homebrew/Cellar/go/1.19.4/libexec/src/testing/testing.go:1399 +0x378
panic({0x10159b580, 0x140017108a0})
    /opt/homebrew/Cellar/go/1.19.4/libexec/src/runtime/panic.go:884 +0x204
github.com/pokt-network/pocket-core/app.NewInMemoryTendermintNodeProtoWithValidators(0x5?, {0x140014bc000, 0x3015, 0x4000}, {0x0, 0x0, 0x0})
    /pocket-core/app/common_test.go:157 +0x2f4
github.com/pokt-network/pocket-core/app.NewInMemoryTendermintNodeProto(0x14000f3e540?, {0x140014bc000?, 0x20?, 0x1016e1578?})
    /pocket-core/app/common_test.go:128 +0x2c
github.com/pokt-network/pocket-core/app.TestClaimProtoTx.func1(0x14000eb5a00)
    /pocket-core/app/tx_test.go:1277 +0xe4
testing.tRunner(0x14000eb5a00, 0x140012c8230)
    /opt/homebrew/Cellar/go/1.19.4/libexec/src/testing/testing.go:1446 +0x10c
created by testing.(*T).Run
    /opt/homebrew/Cellar/go/1.19.4/libexec/src/testing/testing.go:1493 +0x300
FAIL    github.com/pokt-network/pocket-core/app 441.701s
?       github.com/pokt-network/pocket-core/app/cmd/cli [no test files]
?       github.com/pokt-network/pocket-core/app/cmd/pocket_core [no test files]
ok      github.com/pokt-network/pocket-core/app/cmd/rpc 49.750s
?       github.com/pokt-network/pocket-core/app/cmd/rpc/types   [no test files]
ok      github.com/pokt-network/pocket-core/baseapp 0.211s [no tests to run]
?       github.com/pokt-network/pocket-core/codec   [no test files]
?       github.com/pokt-network/pocket-core/codec/legacy    [no test files]
?       github.com/pokt-network/pocket-core/codec/types [no test files]
ok      github.com/pokt-network/pocket-core/codec/unknownproto  0.128s
ok      github.com/pokt-network/pocket-core/crypto  0.187s
ok      github.com/pokt-network/pocket-core/crypto/keys 5.498s
?       github.com/pokt-network/pocket-core/crypto/keys/keyerror    [no test files]
ok      github.com/pokt-network/pocket-core/crypto/keys/mintkey 1.162s
?       github.com/pokt-network/pocket-core/store   [no test files]
ok      github.com/pokt-network/pocket-core/store/cachekv   4.105s
?       github.com/pokt-network/pocket-core/store/cachemulti    [no test files]
?       github.com/pokt-network/pocket-core/store/dbadapter [no test files]
?       github.com/pokt-network/pocket-core/store/errors    [no test files]
ok      github.com/pokt-network/pocket-core/store/gaskv 0.142s
ok      github.com/pokt-network/pocket-core/store/iavl  0.410s
ok      github.com/pokt-network/pocket-core/store/list  0.446s
ok      github.com/pokt-network/pocket-core/store/prefix    0.150s
?       github.com/pokt-network/pocket-core/store/queue [no test files]
ok      github.com/pokt-network/pocket-core/store/rootmulti 0.501s
?       github.com/pokt-network/pocket-core/store/rootmulti/heightcache [no test files]
ok      github.com/pokt-network/pocket-core/store/tracekv   0.139s
ok      github.com/pokt-network/pocket-core/store/transient 0.267s
ok      github.com/pokt-network/pocket-core/store/types 0.335s
ok      github.com/pokt-network/pocket-core/types   0.521s
ok      github.com/pokt-network/pocket-core/types/module    0.231s
ok      github.com/pokt-network/pocket-core/x/apps  0.286s
?       github.com/pokt-network/pocket-core/x/apps/exported [no test files]
ok      github.com/pokt-network/pocket-core/x/apps/keeper   0.377s
ok      github.com/pokt-network/pocket-core/x/apps/types    0.572s
ok      github.com/pokt-network/pocket-core/x/auth  0.414s
?       github.com/pokt-network/pocket-core/x/auth/exported [no test files]
ok      github.com/pokt-network/pocket-core/x/auth/keeper   0.290s
ok      github.com/pokt-network/pocket-core/x/auth/types    0.242s
?       github.com/pokt-network/pocket-core/x/auth/util [no test files]
?       github.com/pokt-network/pocket-core/x/gov   [no test files]
ok      github.com/pokt-network/pocket-core/x/gov/keeper    0.218s
ok      github.com/pokt-network/pocket-core/x/gov/types 0.202s
ok      github.com/pokt-network/pocket-core/x/nodes 0.211s
?       github.com/pokt-network/pocket-core/x/nodes/exported    [no test files]
ok      github.com/pokt-network/pocket-core/x/nodes/keeper  0.276s
ok      github.com/pokt-network/pocket-core/x/nodes/types   0.319s
ok      github.com/pokt-network/pocket-core/x/pocketcore    0.756s
ok      github.com/pokt-network/pocket-core/x/pocketcore/keeper 4.785s
ok      github.com/pokt-network/pocket-core/x/pocketcore/types  1.145s
FAIL

Expected behavior Tests pass.

Operating System or Platform: OSX

nodiesBlade commented 1 year ago

fyi this was a test that was failing prior to v0.9 as well

might even date further back

oten91 commented 1 year ago

please run tests with parallel flag = 1 (-p 1) like this go test -short -v -p 1 ./... 

    --- FAIL: TestClaimProtoTx/claim_tx_from_a_proto_with_proto_codec (0.28s)
panic: failed to listen on 0.0.0.0:36657: listen tcp 0.0.0.0:36657: bind: address already in use [recovered]
    panic: failed to listen on 0.0.0.0:36657: listen tcp 0.0.0.0:36657: bind: address already in use

^ this shows that if tests are executed in parallel the address bindings overlaps

lookfirst commented 1 year ago

@oten91 This seems to work, but woah, it is slow af. I'm on a top of the line m1 max with 64 gigs of ram and it is taking forever to run all the tests. I don't think this is a good solution.

oten91 commented 1 year ago

Yeah, it takes around 10 minutes. They way most test are constructed they usually spawn a node and expects a result, parallel execution may change the expected result if shared between multiple tests. One possible solution is to randomize the ports bindings for each execution and client queries.

lookfirst commented 1 year ago

What's the benefit of spawning a node each time vs. say ... architecting the tests to just call into the node's api directly and not spawn anything?