Generate test result from cargo test

[petrikvladimir]

Calling colcon test should generate xml report with information about passed/failed tests. Otherwise colcon test-result --all is not able to give information about rust tests.

The only solution I found is:

• install a modified version of cargo-test-junit (https://github.com/voop/cargo-test-junit)
• modify colcon-cargo to call cargo test-junit (https://github.com/voop/colcon-cargo/commit/d24ce169ef5229ad210df63ebacb55c8977a2a1b)
  • then I was able to setup travis CI: (https://github.com/voop/rosrust_tests)

Is there a nicer way how to get test-results ?

[lelongg]

I've never used cargo-test-junit but its unresolved issues doesn't make it look like an attractive dependency.

Is it reliable in your experience ?

Do you plan on having your output feature merged in the original repo ?

[petrikvladimir]

I agree that cargo-test-junit is not an attractive dependency. It seems not maintained, that's why I did not make PR.

In my opinion, the best way to solve it would be to parse cargo test output in python directly.

The goal is to capture test output in python:

$ cargo test  -q | grep "test result:"
test result: ok. 8 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out
test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out
test result: ok. 4 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out

and transform it into 3 (?) xml outputs for unit, doc, and integration tests:

<?xml version='1.0'?>
<testsuites name='unit.xml' errors='0' failures='0' tests='8'>
</testsuites>

I guess that would be the minimum requirement for parsing outputs by colcon test-result. Optionally, we can also parse individual test cases from the cargo test output.

Does this solution sound good to you? I can try to implement it in python during the weekend.

[lelongg]

I dug a bit around cargo test output and it looks like it's still a work in progress, so any solution we come with should be considered temporary.

See these threads for reference:

• rust-lang/cargo/issues/1273
• rust-lang/rfcs/issues/816
• rust-lang/rfcs/pull/2318

I'm OK with your solution since it shouldn't add any dependency and do the job while cargo test output becomes more mature.

About the three XML outputs, I'm not aware of JUnit conventions but maybe tests results might land in a single file formatted this way:

<?xml version='1.0'?>
<testsuites name='tests' errors='0' failures='0' tests='8'>
    <testsuite name='unit' errors='0' failures='0' tests='4'></testsuite>
    <testsuite name='doc' errors='0' failures='0' tests='2'></testsuite>
    <testsuite name='integration' errors='0' failures='0' tests='2'></testsuite>
</testsuites>