search

gnolang / gno

Gno: An interpreted, stack-based Go virtual machine to build succinct and composable apps + Gno.land: a blockchain for timeless code and fair open-source
https://gno.land/
Other
841 stars 342 forks source link

Significantly long GnoVM test runtime #1406

Open zivkovicmilos opened 7 months ago

zivkovicmilos commented 7 months ago

Description

Running the gnovm test suite takes an extraordinarily long amount of time to execute:

time make --no-print-directory -C gnovm test  1005.06s user 23.11s system 160% cpu 10:42.20 total

For CI runs, take a look at an example GnoVM CI run: https://github.com/gnolang/gno/actions/runs/7031801537/job/19134177743

Notice that pkg0, pkg1, pkg2take ~13mins to execute in the CI.

An additional problem is that we cannot execute the VM tests in parallel (t.Parallel()), since there are primitives in the VM preventing parallel execution (global vars).

This long test suite execution time takes a large toll on local development and CI workflows.

@ajnavarro @gfanton please add more info to this task if I've missed something, @petar-dambovaliev will look into the issue

ajnavarro commented 7 months ago

Maybe we should make the VM thread-safe, to make it possible to run several of them simultaneously.

zivkovicmilos commented 7 months ago

to make it possible to run several of them simultaneously.

This is actually something that is required down the line, we can't do gas estimations (through an RPC call, cc @MichaelFrazzy) and state transitions at the same time with the current VM

petar-dambovaliev commented 6 months ago

Maybe we should make the VM thread-safe, to make it possible to run several of them simultaneously.

I also lean towards this idea. I made a profile on the slowest tests and the distribution of runtime is kind of equal across the board. In other words, there is no single function bottleneck. The only one that takes more than 2% of the runtime is

// NOTE: for allocation, use *Allocator.NewBlock.
func NewBlock(source BlockNode, parent *Block) *Block

It takes about 7% of the runtime. It is worth improving it. It wouldn't fix the tests runtime, we still need to make them thread safe but it will help a bit with that and also make the general runtime faster.

MichaelFrazzy commented 6 months ago

to make it possible to run several of them simultaneously.

This is actually something that is required down the line, we can't do gas estimations (through an RPC call, cc @MichaelFrazzy) and state transitions at the same time with the current VM

Is this test similar in function to the benchmarking framework that Dylan set up or something that we've had for a while?