search

nspcc-dev / neo-go

Go Node and SDK for the NEO blockchain
MIT License
120 stars 78 forks source link

Profiling/improving our tests #2379

Open roman-khimov opened 2 years ago

roman-khimov commented 2 years ago

We've got a lot of tests which is very good except they take a heck lot of time to run. There were some optimizations already done previously (most notably in #2001), but it's time to make another round. First of all, the most resource-demanding tests need to be identified and then we'll think on whether and how they can be improved.

fyrchik commented 2 years ago

Packages where tests take more than 5 seconds (on my machine):

  1. compiler #2382
  2. cli -- the biggest (80 seconds), designed to perform end-to-end testing, runs blockchain, waits for blocks. Not sure if we can do anything about it.
  3. wallet -- probably stays as it is, most of the time is spent in crypto routines
  4. vm -- TestUT, most of the time spent in execute.
  5. core
roman-khimov commented 2 years ago

cli -- the biggest (80 seconds), designed to perform end-to-end testing, runs blockchain, waits for blocks. Not sure if we can do anything about it.

It can be split into subpackages which can then be run in parallel (the effect was seen in #1472 already).

My current list is:

ok      github.com/nspcc-dev/neo-go/cli 86.593s coverage: 80.0% of statements
...
ok      github.com/nspcc-dev/neo-go/pkg/compiler        37.964s coverage: 92.0% of statements
...
ok      github.com/nspcc-dev/neo-go/pkg/crypto/keys     7.564s  coverage: 80.8% of statements
...
ok      github.com/nspcc-dev/neo-go/pkg/services/notary 5.021s  coverage: 93.0% of statements
ok      github.com/nspcc-dev/neo-go/pkg/services/oracle 13.095s coverage: 74.7% of statements
...
ok      github.com/nspcc-dev/neo-go/pkg/services/stateroot      8.760s  coverage: 84.3% of statements
...
ok      github.com/nspcc-dev/neo-go/pkg/vm      16.205s coverage: 82.0% of statements
...
ok      github.com/nspcc-dev/neo-go/pkg/wallet  9.996s  coverage: 79.0% of statements
roman-khimov commented 1 year ago

It can be split into subpackages

Argh.

?       github.com/nspcc-dev/neo-go/cli [no test files]
ok      github.com/nspcc-dev/neo-go/cli/app     0.036s  coverage: 100.0% of statements
ok      github.com/nspcc-dev/neo-go/cli/cmdargs 0.011s  coverage: 60.8% of statements
ok      github.com/nspcc-dev/neo-go/cli/flags   0.009s  coverage: 97.7% of statements
?       github.com/nspcc-dev/neo-go/cli/input   [no test files]
ok      github.com/nspcc-dev/neo-go/cli/options 0.033s  coverage: 50.0% of statements
?       github.com/nspcc-dev/neo-go/cli/paramcontext    [no test files]
ok      github.com/nspcc-dev/neo-go/cli/query   2.155s  coverage: 45.3% of statements
ok      github.com/nspcc-dev/neo-go/cli/server  1.373s  coverage: 67.8% of statements
ok      github.com/nspcc-dev/neo-go/cli/smartcontract   8.819s  coverage: 94.3% of statements
ok      github.com/nspcc-dev/neo-go/cli/util    0.006s  coverage: 10.9% of statements
?       github.com/nspcc-dev/neo-go/cli/vm      [no test files]
ok      github.com/nspcc-dev/neo-go/cli/wallet  72.103s coverage: 88.2% of statements
roman-khimov commented 1 year ago

Block verification can be disabled in most of tests to remove some crypto overhead.

fyfyrchik commented 9 months ago

Btw, I have recently stumbled upon https://github.com/gotestyourself/gotestsum which is quite useful

$ go test -json -count=1 ./pkg/... | gotestsum tool slowest --threshold 1s
github.com/nspcc-dev/neo-go/pkg/services/rpcsrv TestClient_Wait 22.53s
github.com/nspcc-dev/neo-go/pkg/services/rpcsrv TestSubClientWaitWithMissedEvent 15.21s
github.com/nspcc-dev/neo-go/pkg/services/rpcsrv TestSubClientWait 9.4s
...