anza-xyz / move

Move compiler targeting llvm supported backends
https://discord.gg/wFgfjG9J
Apache License 2.0
108 stars 33 forks source link

Implement unit test runner for Solana VM #324

Closed dmakarov closed 1 year ago

dmakarov commented 1 year ago

Motivation

Move unit tests compiled to Solana architecture should be executed on Solana VM

Test Plan

Added an example module with two unit tests, one should succeed, another expected to fail. More testing in future PRs.

This PR depends on acceptance of #316

dmakarov commented 1 year ago

We can now run stdlib unit tests. I ran some and there are failures.

     Running `target/debug/move test --solana -p language/move-stdlib ascii_tests`
BUILDING MoveStdlib
Running Move unit tests
[ PASS    ] 0x1::ascii_tests::printable_chars_dont_allow_newline
[ PASS    ] 0x1::ascii_tests::printable_chars_dont_allow_tab
[ PASS    ] 0x1::ascii_tests::test_ascii_chars
[ PASS    ] 0x1::ascii_tests::test_ascii_push_char_pop_char
[ PASS    ] 0x1::ascii_tests::test_ascii_push_chars
[ PASS    ] 0x1::ascii_tests::test_invalid_ascii_characters
[ PASS    ] 0x1::ascii_tests::test_nonvisible_chars
[ PASS    ] 0x1::ascii_tests::test_printable_chars
Test result: OK. Total tests: 8; passed: 8; failed: 0
     Running `target/debug/move test --solana -p language/move-stdlib bit_vector_tests`
BUILDING MoveStdlib
Running Move unit tests
[ PASS    ] 0x1::bit_vector_tests::empty_bitvector
[ PASS    ] 0x1::bit_vector_tests::index_bit_out_of_bounds
[ PASS    ] 0x1::bit_vector_tests::longest_sequence_no_set_nonzero_index
[ PASS    ] 0x1::bit_vector_tests::longest_sequence_no_set_zero_index
[ PASS    ] 0x1::bit_vector_tests::longest_sequence_one_set_zero_index
[ PASS    ] 0x1::bit_vector_tests::longest_sequence_two_set_nonzero_index
[ PASS    ] 0x1::bit_vector_tests::longest_sequence_with_break
[ PASS    ] 0x1::bit_vector_tests::set_bit_out_of_bounds
[ PASS    ] 0x1::bit_vector_tests::shift_left_at_size
[ PASS    ] 0x1::bit_vector_tests::shift_left_more_than_size
[ PASS    ] 0x1::bit_vector_tests::single_bit_bitvector
[ PASS    ] 0x1::bit_vector_tests::test_set_bit_and_index_basic
thread '<unnamed>' panicked at 'internal error: entered unreachable code: Unexpected (exp, exit reason) pair: (None, Failure). This should not have happened.'
     Running `target/debug/move test --solana -p language/move-stdlib bcs_tests`
BUILDING MoveStdlib
Running Move unit tests
[ FAIL    ] 0x1::bcs_tests::bcs_address
[ PASS    ] 0x1::bcs_tests::bcs_bool
[ PASS    ] 0x1::bcs_tests::bcs_u128
[ PASS    ] 0x1::bcs_tests::bcs_u16
[ PASS    ] 0x1::bcs_tests::bcs_u256
[ PASS    ] 0x1::bcs_tests::bcs_u32
[ PASS    ] 0x1::bcs_tests::bcs_u64
[ PASS    ] 0x1::bcs_tests::bcs_u8
[ FAIL    ] 0x1::bcs_tests::bcs_vec_u8
thread '<unnamed>' panicked at 'internal error: entered unreachable code: Unexpected (exp, exit reason) pair: (None, Failure). This should not have happened.'
dmakarov commented 1 year ago

@brson @ksolana @nvjle This PR looks like a lot of code, but it's relatively simple. The run_for_unit_test receives Move model environment with the code of all dependencies of a unit test function, id of the module where the unit test function is defined, and the id of a unit test function. The function run_for_unit_test then generates an input.json file that will be used to set up parameters of a program that Solana VM will execute. Move unit tests either don't take any parameters, or take only parameters of type signer. In this PR I didn't add support for unit tests with parameters. All stdlib unit tests don't take any parameters. Then, run_for_unit_test calls the compiler to generate a loadable .so file that contains the unit test function and everything it depends on. The name of the unit test function is passed to the compiler as an option. The code that generates .so file entrypoint adds the unit test function to the list of entry functions. It doesn't matter if the module contains other entry functions, only the unit test function will be called by entrypoint because the unit test function is added to the input.json, which will be used to determine what function the entrypoint will pass the control to. The rest of the code is runner, which is pretty much a copy of what we have in rbpf-tests.rs to create a VM instance and run a program on it. The code in test_runner.rs is processing the result of running the VM and preparing a test report, i.e. whether a test has passed or failed, what was the reason for failure, how many instructions were executed (gas) and how much time it took.

Since multiple instances of the compiler don't seem to work concurrently because of the LLVM library, I enforced sequential execution of all the unit tests. This works ok with using the same file name for input.json and output.so. Each unit test overwrites these files. In a follow-up PR I'm going to change this to use unique names (or paths) for these files for each unit test.