Tests are skipped if enough data is written to stdout

[roxgib]

TL;DR tests should fail if they exit because of excessive output.

I'm mentoring a student on the Rust track who submitted this code for the lasagna exercise:

pub fn remaining_minutes_in_oven(actual_minutes_in_oven: i32) -> i32 {
    let minutes_left_in_oven = expected_minutes_in_oven() - actual_minutes_in_oven;
     while actual_minutes_in_oven <= 40 {
         println!("The remaining minutes in the oven is: {} ", minutes_left_in_oven);
     }
    return minutes_left_in_oven
}

Of course this code will loop forever, printing each time. After some discussion it seems that a test is skipped if enough data is written to stdout. For this exercise Exercism shows "five tests passed" when there are six tests for this exercise. If you simply loop without printing anything the tests fail with a timeout as expected. I've reproduced this with a different exercise in Rust, but I assume it affect any language capable of writing fast enough to beat the timeout.

It's not uncommon for a student to accidentally submit code that gets stuck in a loop, and it'll be confusing for them if the tests pass just because they happen to have a print statement in there.

Edit: BTW I don't recommend running the above code locally for obvious reasons!

[github-actions[bot]]

Hi and welcome to Exercism! :wave:

Thanks for opening an issue :slightly_smiling_face:

• If you are suggesting a new feature or an improvement to Exercism, please take a read of this post, which will likely result in a faster response.
• If you are reporting a bug in the website, thank you! We are getting a lot of reports at the moment (which is great), but we triage and reply as soon as we can.
• If you are requesting support, someone will help shortly.
• For everything else, we will reply or triage your issue to the right repository soon.

[iHiD]

@roxgib Thanks. This is a bug in the rust test runner so I've moved this there. Feel free to PR a fix if you fancy digging into the code.

@ErikSchierboom Could you tag it up appropriately pls (you might also like to fix it if you fancy some Rust and @roxgib doesn't 🙂 ).

[roxgib]

@iHiD Thanks, I'll dig into it a bit today and see if it's something I can fix

[sudhackar]

I found the same problem with a solution a few days back while mentoring

#[derive(Debug, PartialEq)]
pub enum Error {
    SpanTooLong,
    InvalidDigit(char),
}
pub fn lsp(string_digits: &str, span: usize) -> Result<u64, Error> {
    if span > string_digits.len() { return Err(Error::SpanTooLong); }
    if span == 0 { return Ok(1); }
    for starting_index in 0..string_digits.len() {
        println!("start: {}", starting_index);
        while starting_index < span {
            let mut index = starting_index;
            print!("index: {} ", index);
            index += 1;
        }
    }
    Ok(0)
}

There are more than 4 tests

[ZloeSabo]

Hey everyone, I've put together #55 to fix the issue. Could someone please take a look once you have a chance?