informalsystems / atomkraft

Advanced fuzzing via Model Based Testing for Cosmos blockchains
Apache License 2.0
78 stars 10 forks source link

Combine multiple trace tests into single file #168

Closed hvanz closed 1 year ago

hvanz commented 1 year ago

When one runs time atomkraft test trace --path <dir-path>/ ... where <dir-path> is a directory that contains many trace files, Atomkraft generates one python test file for each trace. A test file is simple 3 lines of code:

@itf(<path-to-itf-json-file>, keypath=<keypath>)
def test_trace():
    logging.info("Successfully executed trace " + <path-to-itf-json-file>)

A better approach would be to combine all tests in each subdirectory of <dir-path> into a single file:

@itf(<path-to-itf-json-file1>, keypath=<keypath>)
def test_<path-to-itf-json-file1>():
    logging.info("Successfully executed trace " + <path-to-itf-json-file1>)

@itf(<path-to-itf-json-file2>, keypath=<keypath>)
def test_<path-to-itf-json-file2>:
    logging.info("Successfully executed trace " + <path-to-itf-json-file2>)

This would allow the user to re-execute all the traces in a subdirectory with a single pytest command.